New opportunity

Senior Associate/Assistant Vice President, Agent Assurance Engineer

Temasek · Singapore

About this role

The Agent Assurance Engineer is a specialized role focused on the verification, validation, and continuous monitoring of AI agents within Temasek's investment workflows. This position bridges the gap between technical engineering and governance by ensuring agents operate reliably, consistently, and within defined boundaries. Responsibilities include designing agent harnesses, managing context injection, developing behavioural specifications, and implementing robust evaluation frameworks. The role requires systematic testing, including red-teaming and adversarial evaluation, to identify vulnerabilities and ensure agents meet performance benchmarks before production deployment. Requirements: 4–7 years of experience in AI/ML engineering, AI safety research, or AI quality assurance, with specific expertise in evaluating LLM or agentic systems. Candidates must demonstrate proficiency in prompt engineering, context architecture, and AI observability. Technical skills include deep familiarity with LLM APIs (Anthropic/OpenAI), orchestration frameworks (LangChain, LangGraph, AutoGen), and evaluation tools (LangSmith, RAGAS). Strong Python proficiency is essential. Experience in regulated or high-stakes environments is strongly preferred. The ideal candidate possesses an adversarial mindset, a rigorous scientific approach to testing, and the ability to collaborate effectively with product teams to improve agent safety and quality.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

LLM evaluationPrompt engineeringAI red-teamingAdversarial testingAI system monitoringAnthropic Claude APIOpenAI APILangChainLangGraphAutoGenLangSmith EvalsRAGASDeepEvalLangFuseHeliconeWeights & BiasesPythonpandasnumpyConstitutional AIChain-of-thought elicitation

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Design and Implementation of Agent Harnesses
The role requires building the scaffolding for AI agents, including system prompts, context injection patterns, and memory structures. You must ensure these harnesses effectively constrain agent behavior within defined investment workflow boundaries while maintaining operational flexibility.
Development of Robust Evaluation Frameworks
You are responsible for creating systematic testing suites that assess task completion, tool-calling accuracy, and instruction-following fidelity. This involves building ground-truth datasets and scoring rubrics to enforce strict performance benchmarks before any production deployment.
Adversarial Testing and Red-Teaming
A core priority is proactively probing agents for vulnerabilities, such as prompt injection or out-of-scope behaviors. You must apply an adversarial mindset to identify failure modes and ensure the system remains resilient against ambiguous or malicious inputs.
Continuous Monitoring and Observability
You must implement and manage observability platforms to track agent performance in real-time. This includes detecting agent drift, managing failures, and using production feedback loops to iteratively improve agent safety and quality.

Likely challenges to think through

Balancing Innovation with Governance
As an assurance engineer in a high-stakes investment environment, you will face the tension between enabling rapid AI deployment and enforcing rigorous safety gates. Prepare by demonstrating how you can act as a collaborative partner to product teams without compromising on necessary risk management and compliance standards.
Defining Behavioral Specifications for Complex Workflows
Translating abstract investment goals into concrete, testable behavioral specifications is difficult. Prepare by articulating a methodology for defining 'ground truth' in ambiguous scenarios and how you would handle edge cases where agent decision-making boundaries are not clearly defined.
Maintaining Reliability in Evolving AI Systems
AI agents are non-deterministic, making regression testing and version control complex. Prepare to discuss how you manage prompt versioning and ensure that updates to models or orchestration frameworks (like LangGraph or AutoGen) do not introduce regressions in agent behavior.

Practice your answers

Technical practice

Designing Agent Harnesses and Context Injection

What to coverExplain your approach to building agent harnesses that enforce behavioural constraints. Discuss how you manage context prioritization, filtering, and memory structures to ensure agents operate within defined boundaries during multi-turn interactions.

LLM Evaluation Frameworks and Metrics

What to coverDescribe how you design evaluation suites for agentic systems. Focus on how you define ground-truth labels, scoring rubrics for tool-calling correctness, and how you utilize tools like RAGAS or LangSmith to measure instruction-following fidelity.

Adversarial Testing and Red-Teaming

What to coverHow do you systematically probe an AI agent for prompt injection vulnerabilities or out-of-scope behaviours? Detail your methodology for creating adversarial datasets and assessing agent performance under conflicting or ambiguous inputs.

AI Observability and Anomaly Detection

What to coverDiscuss your experience implementing observability platforms like LangFuse or Helicone in production. How do you define and detect 'agent drift' or behavioural anomalies in real-time investment workflows?

Prompt Versioning and Change Management

What to coverExplain the infrastructure required to manage prompt versioning in a production environment. How do you ensure that changes to system prompts or model versions do not introduce regressions in agent behaviour?

Constitutional AI and Behaviour Steering

What to coverHow have you applied techniques like constitutional AI or chain-of-thought elicitation to steer agent behaviour? Discuss the trade-offs between these methods and traditional fine-tuning for safety and alignment.

Python for Evaluation Automation

What to coverDescribe a scenario where you used Python, pandas, or numpy to automate the analysis of agent evaluation results. How do you ensure statistical rigour when interpreting large-scale test outputs?

Orchestration Frameworks (LangChain/LangGraph/AutoGen)

What to coverCompare the use of LangChain, LangGraph, and AutoGen for building complex agentic workflows. Which framework do you prefer for implementing strict tool permission scopes and why?

Behavioural practice

Collaborating with Product Teams on Safety

What to coverDescribe a time you had to communicate a critical vulnerability or safety concern to a product team. How did you balance the need for rigorous security controls with the team's delivery timelines?

Handling Ambiguous Requirements in High-Stakes Environments

What to coverProvide an example of how you defined behavioural specifications for an agent when the requirements were initially vague. How did you ensure the final implementation met the necessary governance standards?

Continuous Learning in AI Safety

What to coverThe field of AI evaluation evolves rapidly. Describe a recent research paper or emerging technique you learned and how you evaluated its potential application to your previous work.

Managing Agent Failures in Production

What to coverTell me about a time an AI system failed in production. What was your process for identifying the root cause, implementing a fix, and ensuring the failure did not recur?

Sharpen your knowledge

Topics to prepare

Agent Behavioural Specifications
Understand how to translate high-level business requirements into formal, testable behavioural specifications. Exercise: Draft a specification document for a hypothetical investment research agent that defines specific tool-use boundaries, prohibited actions, and escalation triggers. This is critical for establishing the ground truth against which agent performance is measured.
Adversarial Evaluation and Red-Teaming
Study methodologies for probing LLM-based systems for vulnerabilities such as prompt injection, jailbreaking, and out-of-scope reasoning. Exercise: Design a red-teaming plan for a multi-turn agent, creating a set of adversarial prompts that test the agent's adherence to safety constraints under ambiguous or conflicting inputs.
AI Observability and Drift Detection
Learn the principles of monitoring production AI systems for performance degradation and behavioural drift. Exercise: Map out a monitoring architecture using tools like LangFuse or Helicone to track latency, token usage, and semantic drift in agent responses, ensuring you can detect when an agent deviates from its intended operational scope.

Skills to revise

Evaluation Framework Implementation
Gain hands-on experience with automated evaluation tools to measure agent accuracy and instruction-following. Exercise: Use RAGAS or LangSmith Evals to build a test suite that evaluates a RAG-based agent's retrieval precision and response faithfulness against a small, curated golden dataset.
Context Architecture and Prompt Engineering
Develop expertise in managing agent context, including memory structures and system prompt design. Exercise: Build a prototype agent using LangGraph that implements a structured memory management system, demonstrating how to prioritize and filter context to maintain agent focus during complex, multi-turn investment workflows.
Python-based Data Analysis for AI Assurance
Utilize Python libraries to analyze evaluation results and identify failure patterns. Exercise: Write a Python script using pandas and numpy to process a large log file of agent interactions, calculating success rates and identifying specific edge cases where the agent failed to follow system instructions.

Your preparation checklist

Develop an agent evaluation framework project
Prepare a technical walkthrough of a custom evaluation pipeline you have built using tools like LangSmith, RAGAS, or DeepEval. Demonstrate how you defined ground-truth datasets, scoring rubrics for tool-calling accuracy, and how you automated regression testing for prompt changes.
Demonstrate adversarial testing methodology
Prepare a case study detailing a red-teaming exercise where you identified vulnerabilities in an LLM-based system. Explain your process for probing prompt injection, boundary condition failures, and how you translated those findings into actionable remediation strategies.
Showcase context architecture expertise
Prepare a technical explanation of how you have designed agent harnesses in previous roles. Focus on how you managed context injection, implemented memory structures, and used techniques like chain-of-thought elicitation or structured output enforcement to constrain agent behavior.
Prepare for discussions on AI observability
Be ready to discuss your experience with monitoring tools like LangFuse, Helicone, or Weights & Biases. Explain how you have implemented structured logging to detect agent drift and manage failures in high-stakes production environments.
Formulate questions on governance and risk
Prepare thoughtful questions regarding how Temasek balances rapid AI deployment with their specific governance requirements. Ask about their current process for defining 'behavioral specifications' and how they handle escalation triggers when an agent operates out-of-scope.

Availability last checked: 18 Oct 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in Singapore. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…