About this role
Drive operational excellence and governance for agentic applications across the bank, managing agent lifecycle, evaluation, and responsible AI practices. Partner with cross-functional teams to implement, monitor, fine-tune prompts and LLM integrations, and ensure secure, efficient, and compliant agent operations.
Skills for this role
Performance MonitoringProcess ImprovementService Lifecycle ManagementAgent lifecycle managementAgent evaluationPrompt engineeringPrompt fine-tuningSystem message designLLM fine-tuningRAGEvaluation frameworksPythonExperimental designMeasurementError analysisStatistical thinkingEmbeddingsRetrieval methodsClassificationInformation extractionCalibrationReliability testingResponsible AIBias testingPrivacy leakage testingAudit artifactsLogging and audit trailsRoot cause analysisAutomation