About this role
The AI Engineer will build and tune classifiers and guardrail evaluation systems to keep Realtor.com's LLM-powered assistant compliant, safe, and helpful. The role owns the AI safety layer end-to-end, including dataset curation and evaluation, prompt-injection/jailbreak defenses, guardrail infrastructure as code, red-teaming, and partnering with ML, backend, product, and legal stakeholders to ensure safe launches.
Skills for this role
AI safetyAI evaluationLLM applicationsPythonClassification metrics (precision/recall/F1)Confusion matrix analysisDataset curationDataset calibrationPrompt engineeringLLM classifier tuningDeepEvalRAGASArize PhoenixLangSmithOpenAI EvalsDataset synthesisPrompt optimizationPrompt-injection defenseJailbreak defenseOWASP LLM Top 10Input/output filteringLeast-privilege tool accessAdversarial testingGoogle Cloud Model ArmorAWS Bedrock GuardrailsAzure AI Content SafetyTerraformInfrastructure as CodeGoogle Cloud Vertex AIGemini (Google) / Gemini models or tooling? (listed as Gemini)