New opportunity

AI Lead Developer, AI Products

Codex Solutions · Singapore

About this role

Lead the development of scalable AI products for Home Team use cases in Singapore. The role covers LLM-based applications, Retrieval-Augmented Generation (RAG), agentic AI and multimodal AI, from data preparation and model development through production deployment and improvement.

Responsibilities

Design, develop, integrate and optimise AI/ML solutions. Handle data preprocessing, engineering and annotation for model training and development.
Work with software engineers to write production-quality code and deploy AI products on Cloud Platform (GCC) or on-premises infrastructure. Collaborate with Product Management to translate product requirements into machine learning and deep learning model requirements.
Design model evaluation and monitoring; improve models using pre-training and post-training techniques; and optimise inference performance and resource use. Implement guardrails and other risk mitigations for safe, responsible AI use in line with organisational and regulatory requirements.
Troubleshoot complex production issues, improve reliability and operational performance, provide technical advice, review technical articles, and share knowledge across multidisciplinary teams.

Requirements

A postgraduate or tertiary qualification in Computer Science, Computer Engineering, Data Science, Data Analytics, Machine Learning or a relevant field.
At least 6 years of software engineering and AI engineering experience, including at least 3 years developing and deploying production AI applications.
Strong Python and production software development skills; hands-on experience building, deploying and maintaining AI/ML solutions. Experience with LangChain or LangGraph, open models such as Llama 4 and GPT OSS, scalable applications, Docker, REST APIs, Kubernetes, vector databases and model serving frameworks.
Ability to collaborate across teams, communicate technical concepts and demonstrate solutions to stakeholders with varying technical backgrounds, and work independently. Knowledge of emerging AI technologies is expected. Homeland Security domain knowledge is an added advantage.

This is a permanent role. The posting offers public-sector digital work, career development, technical upskilling, and access to modern tools and cloud technologies. Selected candidates may need to undergo security clearance for classified or sensitive projects.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

PythonMachine LearningDeep LearningLarge Language ModelsRetrieval-Augmented GenerationAgentic AIMultimodal AILangChainLangGraphLlama 4GPT OSSDockerKubernetesREST APIsVector DatabasesModel ServingModel EvaluationModel MonitoringModel TrainingData PreprocessingData AnnotationModel Inference OptimizationAI GuardrailsCloud PlatformsSoftware EngineeringProduction TroubleshootingCross-functional CollaborationTechnical Communication

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Deliver AI products end to end
The role spans data preprocessing and annotation, model development, and deployment of scalable LLM, RAG, agentic and multimodal solutions for Home Team use cases.
Build production-ready systems
Work with software engineers on production-quality Python, REST APIs and containerised applications deployed through Kubernetes on Cloud Platform (GCC) or on-premises infrastructure.
Evaluate and improve deployed models
Design model evaluation and monitoring, apply pre-training and post-training improvements, and optimise inference performance and resource use.
Embed responsible AI safeguards
Develop guardrails and other risk mitigations aligned with organisational standards and regulatory requirements.
Provide cross-functional technical leadership
Translate product needs into model requirements, advise on technical decisions, troubleshoot complex production issues, and communicate solutions across multidisciplinary teams.

Likely challenges to think through

Choosing the right AI approach
Preparation hypothesis: LLM, RAG, agentic and multimodal options may differ in complexity, performance and operational cost. Prepare to explain how you would compare approaches against a product requirement and evaluation results.
Balancing inference quality and resources
Preparation hypothesis: Model improvements may increase latency or resource use. Prepare an example of how you would measure quality alongside inference performance, then decide what to optimise.
Deploying safely across environments
Preparation hypothesis: Supporting GCC or on-premises deployment while applying guardrails may complicate reliable operation. Prepare to discuss deployment and monitoring choices, failure handling, and how you would check that safeguards remain effective.

Practice your answers

Technical practice

How would you translate a Home Team product requirement into an AI solution design?

What to coverSuggested preparation: Explain how you would clarify the intended use, data constraints and measurable outcomes with Product Management before choosing an ML, deep learning or LLM approach. Describe how those decisions become model and production requirements.

How would you design and evaluate a scalable RAG application?

What to coverSuggested preparation: Cover document preparation, retrieval with a vector database, generation and how you would test whether answers are supported by retrieved material. Discuss trade-offs among retrieval quality, latency and resource use.

When would you use an agentic workflow rather than a simpler LLM application?

What to coverSuggested preparation: Describe what additional steps or decisions justify an agentic design and how you would constrain and observe its behaviour. If discussing LangChain or LangGraph, explain the design choice rather than assuming a particular framework is required for every use case.

How would you approach a multimodal AI use case?

What to coverSuggested preparation: Explain how you would establish which input types and outputs the use case actually needs, then prepare suitable data and evaluate performance across them. Address the added integration and reliability trade-offs without assuming a specific modality or project.

How would you prepare and annotate data for model development?

What to coverSuggested preparation: Discuss data quality checks, preprocessing, annotation consistency and how you would separate data for development and evaluation. Explain how errors or inconsistent labels could affect a production AI application.

How would you evaluate, monitor and improve an in-house AI model?

What to coverSuggested preparation: Distinguish pre-deployment evaluation from monitoring after release, using measures tied to the product requirement. Explain when pre-training or post-training work might be justified and how you would verify that a change improves the system.

How would you deploy and maintain a production AI application?

What to coverSuggested preparation: Outline a Python service with a REST API, containerisation with Docker, and deployment considerations for Kubernetes on Cloud Platform (GCC) or on-premises infrastructure. Address model serving, integration with software engineers and how you would check operational reliability.

How would you reduce inference latency and resource use?

What to coverSuggested preparation: Describe how you would measure bottlenecks across the application and model-serving path before optimising. Discuss how performance changes could affect output quality, capacity and operating cost.

How would you design guardrails for a production AI product?

What to coverSuggested preparation: Start with the risks of the intended use and explain how you would test and monitor mitigations. Relate the design to organisational standards and regulatory requirements without assuming a particular policy or regulation.

How would you diagnose a complex production AI failure?

What to coverSuggested preparation: Explain how you would narrow the issue across data, retrieval, model behaviour, serving and application integration. Describe how you would validate a fix and improve monitoring or reliability to reduce recurrence.

Behavioural practice

Tell me about a time you turned an ambiguous product need into technical requirements.

What to coverSuggested preparation: Use a real situation to explain how you worked with product stakeholders, what questions you asked and what decisions you made. State the resulting requirement or outcome without inventing metrics.

Tell me about a time you delivered an AI solution with a multidisciplinary team.

What to coverSuggested preparation: Set out the situation, your specific actions with engineering or other partners, and the result. Focus on how you handled differing priorities while keeping the work production-ready.

Tell me about a time you explained or demonstrated a technical solution to a non-technical audience.

What to coverSuggested preparation: Describe the audience and decision at stake, how you adapted the explanation or demo, and what happened afterward. Show how you communicated limitations as well as capabilities.

Tell me about a time you took ownership of a difficult production issue.

What to coverSuggested preparation: Give the situation, your diagnostic and coordination actions, and the result. Distinguish the immediate remedy from any longer-term reliability improvement.

Tell me about a time you assessed a new AI approach and shared your recommendation.

What to coverSuggested preparation: Explain the question you were asked to assess, the evidence or technical material you reviewed, and how you communicated the trade-offs. Describe the resulting decision or learning, including uncertainty where relevant.

Sharpen your knowledge

Topics to prepare

LLM application architectures
Posting: The role covers LLM-based applications, RAG, agentic AI and multimodal AI. Revision exercise (suggestion): For one hypothetical Home Team use case, compare when retrieval, an agentic workflow or multimodal input would be appropriate, and identify a limitation of each.
Model development and improvement
Posting: The role includes data preparation, model development, and improvement through pre-training and post-training techniques. Revision exercise (suggestion): Sketch how data quality, annotation and the choice between pre-training and post-training would affect an AI product’s development plan.
Evaluation and monitoring
Posting: The developer designs model evaluation and monitoring for in-house AI solutions. Revision exercise (suggestion): Draft an evaluation plan for a RAG application, separating checks before deployment from signals to monitor in production.
Scalability and inference trade-offs
Posting: The role requires scalable AI applications and optimisation of inference performance and resource use. Revision exercise (suggestion): Walk through a hypothetical increase in request volume and explain how you would investigate latency, resource use and reliability before choosing an improvement.
Responsible AI and product requirements
Posting: The developer translates product requirements into model requirements and implements guardrails aligned with organisational and regulatory requirements. Revision exercise (suggestion): Turn a hypothetical product request into model acceptance criteria, then identify risks, proposed guardrails and questions requiring organisational guidance.

Skills to revise

Build a Python RAG workflow
Posting: Strong production Python skills and experience with LangChain or LangGraph and vector databases are requested. Revision exercise (suggestion): Build a small Python RAG prototype using a vector database and one named framework; test retrieval failures and explain how you would make the code maintainable.
Prepare and annotate development data
Posting: Responsibilities include data preprocessing, engineering and annotation. Revision exercise (suggestion): Prepare a small, non-sensitive sample dataset, document cleaning and annotation decisions, and inspect inconsistent examples before model development.
Package and serve an AI application
Posting: The requirements name Docker, REST APIs, Kubernetes and model serving frameworks; deployment may be on Cloud Platform (GCC) or on-premises infrastructure. Revision exercise (suggestion): Containerise a small AI service with a REST endpoint, outline its Kubernetes deployment, and explain what you would verify in either hosting environment.
Evaluate and troubleshoot production behaviour
Posting: The role includes evaluation, monitoring, complex production troubleshooting and reliability improvements. Revision exercise (suggestion): Run a repeatable evaluation of a prototype, inspect a deliberately failing response, and document the evidence and checks you would use to isolate the cause.
Optimise model inference
Posting: Optimising inference performance and resource utilisation is an explicit responsibility. Revision exercise (suggestion): Benchmark a small model-serving setup under two request loads, record latency and resource use, and justify one change against those measurements.
Explain technical decisions to stakeholders
Posting: The role calls for cross-functional collaboration, technical advice and clear demos for audiences with varied technical backgrounds. Revision exercise (suggestion): Prepare a brief demo of a prototype and give two explanations of an evaluation result: one for engineers and one for Product Management.

Your preparation checklist

Map the stated requirements to evidence
Prepare a one-page record of your qualification and relevant software and AI engineering work, including dates and production deployments. Identify specific evidence for Python, LangChain or LangGraph, open models, Docker, REST APIs, Kubernetes, vector databases and model serving; mark gaps rather than inventing examples.
Build a production AI project walkthrough
If you have a suitable project, prepare a diagram and a short account of the problem, your decisions, data preparation, model or application design, deployment, results and your own contribution. Highlight RAG, agentic or multimodal components only where they were actually used.
Practise a Home Team design exercise
As preparation, sketch a hypothetical, non-sensitive use case: translate a product need into model requirements, compare an LLM/RAG approach with alternatives, and explain data engineering, annotation and integration choices. Label it as an exercise, not a Codex Solutions project.
Draw a deployment and scaling plan
Prepare a production architecture showing Python services, REST APIs, containers, Kubernetes, vector storage and model serving where appropriate. Explain how the design might differ between Cloud Platform (GCC) and on-premises deployment; these are both named in the posting.
Prepare an evaluation and safety plan
Define example quality tests, monitoring signals, guardrails, and inference latency and resource measures for your design exercise. Be ready to explain how findings would guide post-training, optimisation or other improvements while meeting organisational and regulatory requirements.
Rehearse troubleshooting and stakeholder communication
Prepare one genuine production-issue example, if available, covering diagnosis, reliability improvements and what you communicated to engineers and non-technical stakeholders. Practise a brief demo or explanation without disclosing sensitive information.
Write targeted questions for the employer
Ask which Home Team use cases are priorities, how Product Management defines success, what evaluation and guardrail standards apply, and how teams choose between GCC and on-premises deployment. You can also ask how work involving restricted information affects the role.

Availability last checked: 4 Nov 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in Singapore. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…