About this role
Lead scaling, governance, and operationalization of Generative AI and LLM solutions by owning AI quality standards, prompt and model versioning, observability, safety, incident response, and cross-functional coordination; develop LLMOps playbooks and evaluation frameworks for Ask Sovi products.
Skills for this role
LLMOpsGenerative AIAI qualityEvaluation frameworksPrompt versioningModel versioningRegression detectionAI governanceAI safetyGuardrailsRed Team testingObservabilityLangfuseLangSmithAI evaluation toolsLatency monitoringCost monitoringQuality metricsSafety metricsJiraTicket triageCross-functional coordinationLLMOps playbooksIncident responseTroubleshootingRoot cause analysisContent moderationAudit loggingRelease managementStakeholder management and communication''Hallucination detection''Model drift