About this role
Design and build multi-dimensional evaluation metric systems; conduct red-team stress testing and safety control to identify bias, hallucinations and policy-violating content; perform root-cause analysis and collaborate with algorithm engineers for prompt optimization and fine-tuning; build automated evaluation pipelines using LLM-as-a-Judge and research user experience in social/companion products.
Skills for this role
Evaluation metric designRed-team testingSafety controlStress testingBias detectionHallucination analysisContent policy complianceValues alignmentRoot-cause analysisPrompt optimizationModel fine-tuningLLM-as-a-JudgeLLMsGenerative AIAutomated evaluation pipelinesAutomated data production pipelinesConventional evaluation methodsAutomated weakness miningSelf-InstructEvol-InstructSynthetic Data RLSFTRLHFPythonShell scriptingLinuxUser experience researchMultilingual models