About this role
Member of Amazon's AGI team responsible for developing quality strategies, auditing frameworks, and automated evaluation systems for Amazon Nova LLM and multimodal models; conduct manual and meta-audits, design LLM-as-a-Judge systems and evaluation rubrics, and configure data collection workflows.
Skills for this role
Machine LearningStatisticsQuality AssuranceAuditing MethodologiesAutomated Evaluation SystemsLarge Language Models (LLMs)Multimodal SystemsDesigning SOPsQuality MetricsSampling MethodologiesManual AuditingMeta-auditLLM-as-a-Judge SystemsEvaluation RubricsAutomated Quality AssessmentData Collection WorkflowsJavaC++PythonSQLRDBMSData WarehouseAlgorithm ImplementationResearch Publications