About this role
Google’s Trust and Safety SAGE team seeks a Senior Engineering Analyst for AI Answers in Google Search. The role leads the architecture and scaling of automated safety evaluation and enforcement for generative AI products. It includes moving from bespoke evaluation scripts to high-throughput, low-latency LLM-as-a-judge autoraters that support continuous offline testing and real-time launch decisions. The work spans sensitive areas such as health and clinical accuracy, civics and elections, misinformation, and AI ecosystem risks.
Responsibilities
- Establish metrics and benchmarks for generative AI issues, quantify improvements, develop datasets, and run evaluations to help engineers improve Search, Labs, and Assistant products.
- Design, develop, and scale production-grade autoraters and automated evaluation systems for launch decisions and continuous regression testing. Support safety guardrails and enforcement mitigations targeted at under 100 milliseconds.
- Apply technical judgment to project delivery, assess proposals and risks, and manage interdependent projects with compressed timelines. Improve tools and operational workflows through analysis, presentations, process changes, and automation.
- Partner with Product, Search Core, and GDM engineering teams on technical roadmaps, integrations, and cross-product risk registers. The role may involve exposure to graphic, controversial, or upsetting content.
Minimum qualifications: A bachelor’s degree or equivalent practical experience; 5 years of data analysis experience identifying trends, producing summary statistics, and drawing insights from quantitative and qualitative data; 5 years in anti-abuse work involving signal analysis, model development, data analysis, and cybersecurity; and programming experience in a language such as SQL, Python, or Golang.
Preferred qualifications
Experience with fraud or abuse investigations, risk management, security, and threat analysis for communication products; end-to-end encrypted communications and anti-abuse messaging solutions; large-scale analysis and ML reputation models; policy and enforcement models addressing harmful actor and behavior patterns; and privacy- and safety-focused product design. Strong problem-solving, critical thinking, communication, and ability to influence stakeholders and senior leadership are also preferred.
Applicants can indicate a preferred location among Seattle, Atlanta, Austin, Kirkland, and Washington D.C. US pay is $159,000–$230,000, plus a 15% bonus target, equity, and benefits. Eligible US employees have access to insurance, retirement benefits, paid time off, leave, and holidays.