About this role
Build internal systems and tools to improve developer experience for LLMs, data, and experimentation, including prompt management, evaluation, benchmarking, and data pipelines; partner with engineers, data scientists, and researchers to streamline prompting, fine-tuning, deployment, and production reliability.
Skills for this role
PythonBackend developmentInternal toolingLarge Language Models (LLMs)OpenAI APIAnthropic APIPrompt managementExperimentation systemsModel evaluationFine-tuningModel deploymentBenchmarkingRegression testingMonitoringData collectionData labelingFeedback loopsData pipelinesObservabilitySystems thinkingDeveloper toolingCommunication