Research Scientist, Societal Impacts

About the Role

As a societal impacts research scientist at Anthropic, you'll join a team conducting empirical research on AI's effects across society. You'll help develop and apply novel measurement systems to understand AI's real-world impact, while working to ensure these powerful technologies benefit humanity.

Our team combines rigorous empirical methods with creative technical approaches. We’re currently grappling with big questions in three key areas on how AI might impact: the economy, peoples’ wellbeing, and education. Additionally, we are continuously studying socio-technical alignment (what values do our systems have?), and evaluating novel AI capabilities as they arise. We develop privacy-preserving tools to measure AI's effects at scale, conduct mixed-methods studies of human-AI interaction, and translate research insights into actionable recommendations for both product and policy.

Note: We are only open to hiring in San Francisco for this team.

Responsibilities

• Design and conduct empirical studies to understand AI's societal effects, ranging from large-scale quantitative analyses using privacy-preserving measurement systems (like Clio) to qualitative investigations of human-AI interaction
• Develop novel methodological approaches for studying AI systems' real-world impacts, including creating new evaluation frameworks, measurement tools, and analysis techniques
• Lead research projects across our key focus areas:
• Economic impact: Studying how AI affects labor markets and the future of work
• Health & wellbeing: Investigating AI's effects on health and human relationships
• Education & epistemics: Examining AI's influence on learning and cognition
• Socio-technical alignment: Ensuring AI systems embody appropriate values and studying how these values relate to real-world behavior
• Technical evaluation: Assessing emerging AI capabilities and developing improved measurement methods
• Collaborate with cross-functional teams to translate research insights into concrete product improvements and policy campaigns
• Communicate findings through research publications, policy briefs, and presentations to diverse stakeholders
• Contribute to the development of measurement infrastructure and evaluation frameworks that enable systematic study of AI's societal effects

You May Be a Good Fit If You Have

• Experience conducting empirical research on socio-technical systems, particularly studies that combine quantitative and qualitative methods
• Strong technical skills, including:
• Proficiency in Python, prompt engineering, and data science 
• Experience building technical evaluations of AI systems
• Fine-tuning large language models
• A desire to do whatever it takes to get the research done, including writing code, debugging code, and pair programming with others
• Track record of leading research projects from conception to publication, including:
• Identifying important research questions
• Designing rigorous studies
• Developing new methodological approaches when needed
• Effectively communicating findings to diverse audiences
• A desire to go above and beyond publications in order to translate research into real world impact
• Expertise in one or more of our focus areas (economics, health/wellbeing, education, alignment, evaluation methods)
• Comfort working with AI systems and ability to think critically about their capabilities and limitations
• Strong interest in ensuring AI development benefits humanity
• Ability to collaborate effectively with teams across technical research, policy, and product development

Some Examples of Our Recent Work

• Clio: Privacy-Preserving Insights into Real-World AI Use
• Measuring the Persuasiveness of Language Models
• Evaluating feature steering: A case study in mitigating social biases
• Collective Constitutional AI: Aligning Language Models with Public Input
• GlobalOpinionQA: Measuring Representation of Global Values in Language Models
• Discrimination in Language Model Decisions: Evaluation and Mitigation
• Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned 

Additional Information

For this role, we're open to candidates interested in either a 6-month research residency or a full-time position. The residency track includes the expectation of conversion to full-time for successful residents.

Deadline to apply: None. Applications are reviewed on a rolling basis.