About this role
Own AI features from prototype through production for LLM-powered products across the patient experience, clinical operations, and internal tooling, building orchestration layers, prompts, retrieval systems, embeddings, agent workflows, and tooling. Design evaluation and observability infrastructure, safety and reliability guardrails, integrate third-party tools, and collaborate with clinical teams to ensure quality, compliance, and operational reliability.
Skills for this role
LLMsPythonBackend servicesPrompt orchestrationTool callingRetrievalRAGEmbeddingsAgent workflowsStructured outputsEvaluation infrastructureRegression testingLLM-as-a-judge evaluationSynthetic datasetsHuman review workflowsQuality metricsObservabilityAccuracy measurementLatency measurementCost measurementHallucination detectionSafety and reliability engineeringComplianceOrchestration frameworksLangChainLangGraphAgentCoreVector databasesPrompt/version managementEvaluation platforms``LangSmith``Braintrust``Arize Phoenix`