@natematiasDesigning Field Experiments at Scale (Seminar)
Undergraduate Class, Princeton University, Sociology, 2019
Online platforms, which monitor and intervene in the lives of billions of people, routinely host thousands of experiments to evaluate policies, test products, and contribute to theory in the social sciences. These experiments are also powerful tools to monitor injustice and govern human and algorithm behavior. How can we do field experiments at scale, reliably, and ethically?
In this hands-on undergraduate class (gradstudents are welcome), students will develop practical experimentation skills, engaging with methods, theory, ethics, and politics of large-scale behavioral research online. For a final project, student teams will develop, conduct, and report on a novel experiment together with an online community or platform.
About The Class
In the spring of 2018 and 2019, I taught this Sociology department seminar class to 10 undergraduates and gradstudents doing majors in political science, operations research, computer science, economics, and philosophy. In 2018, students developed five new field experiments and one replication in the course of a single semester, studies that we successfully launched. In 2019, students developed several more studies, with results forthcoming.
Course materials
Blog posts about the course
Studies developed in the course (more forthcoming)
- (Nov 2018) Matias, J. N. Teaching Large-Scale Digital Experimentation to Undergraduates and Graduate Students. CODE@MIT Conference.
- (Nov 2018) Matias, J.N., Hounsel, A., Hopkins, M. Do Big Social Media Platforms Have Effective Ad Policies? The Atlantic.
- (May 2018) Estimating the Effect of Rule Postings on Community Engagement and Rule Compliance in Subreddits
- (May 2018) Promoting Inclusion and Participation in an Online Feminism Community
- (May 2018) Guiding Question-Asking and Appreciation in an Online Q&A Community (link TBD)