Publications
Looking for data and code? Replication archive →
Books
Explaining Discrimination
[2] Crabtree, Charles and John B. Holbein. Explaining Discrimination. Under advanced contract with Cambridge University Press
Making Young Voters
[1] Holbein, John B. and D. Sunshine Hillygus. Making Young Voters: Converting Civic Attitudes into Civic Action. New York, NY: Cambridge University Press, 2020 ★ Reviewed in: Public Opinion Quarterly, LA Review of Books, Perspectives on Politics, The Journal of Politics
Articles
- [38] Charles Crabtree and John B. Holbein. "How Strong Is the Evidence for Peaceful Resistance? Not Very." Conditionally Accepted, American Political Science Review
Are peaceful social movements really more effective than violent ones? Chenoweth and Stephan's Why Civil Resistance Works argues that nonviolent resistance consistently outperforms violent resistance in achieving democratic outcomes. This conclusion has had enormous influence among academics, political actors, the media, and on-the-ground activists. We find that some results were misreported in ways that make the data appear to support civil resistance when they do not. Moreover, we find that the results are not robust to modest and reasonable changes in model specification, that they are sensitive to omitted variable bias, and that they rely on instrumental variable models with implausible assumptions that yield unstable estimates. Taken together, our replication and extension suggest that the available cross-national evidence provides no clear basis for the claim that nonviolent campaigns are substantially more effective than violent ones. This is not evidence that nonviolence backfires; it is evidence that the strong, unconditional advantage reported in Why Civil Resistance Works is not supported by the data. Our findings underscore the importance of careful methodological choices when drawing broad conclusions about how social movements achieve their goals.
- [37] Charles Crabtree, John B. Holbein, Mitchell Bosley, and Semra Sevi. “Can AI Help Reduce Prejudice? Evaluating the Effectiveness of AI-Powered Personalized Persuasion on Support for Transgender Rights” Forthcoming, PNAS Nexus
Personalized interpersonal conversations are among the most effective known tools for reducing prejudice, yet they are difficult to scale because they require skilled human facilitators. This study tests whether artificial intelligence can approximate the effects of these interventions. Using OpenAI's GPT-4o, we developed a messaging-based intervention that engaged U.S. participants in individualized, morally aligned dialogues about transgender rights. In a preregistered experiment, these AI-mediated conversations significantly increased support for transgender rights across multiple attitudinal measures. Robustness checks, including weighting and sensitivity analyses, confirmed the reliability of these effects, and analyses of the conversations support the idea that moral matching between the AI and participants played a key role in reducing prejudice. However, follow-up data collected one week later indicated that the attitudinal gains diminished over time, suggesting that reinforcement may be necessary to sustain change. Together, these findings indicate that generative AI can facilitate value-aligned dialogue capable of shifting social attitudes, while highlighting the challenge of achieving durable impacts.
- [36] Hans Hassell and John B. Holbein. "Changing the Scope of Conflict? Voter Registration and George Floyd’s Murder" Forthcoming, British Journal of Political Science
Prior work has often studied how focusing events – in particular police killings of unarmed people – affect citizens’ attitudes. Do focusing events also affect citizens’ behaviors – changing the scope of conflict by incorporating prospective voters into the political system? We take as an empirical case the murder of George Floyd. Using nationwide voter file data from the United States, we leverage regression discontinuity in time methods to show that Floyd’s murder noticeably increased voter registration. These effects are present across many individual subgroups, providing evidence of mobilization and counter-mobilization. Floyd’s untimely death sparked many people from previously marginalized backgrounds to register, but simultaneously also increased registrations among the politically advantaged – thus mitigating any impact on the partisan balance of power. In short, high-profile traumatic events like Floyd’s death do not appear to disproportionately advantage groups seeking reforms to change the circumstances surrounding such events.
- [35] Gaddis, S. Michael, Natasha Quadlin, Charles Crabtree, John B. Holbein, and Edvard Nergård Larsen. "Missing Results of Discrimination: A Systematically Comparative Meta-Analysis and Re-Examinations of Racial, Gender, and Intersectional Discrimination using Correspondence Audits" Conditionally Accepted, Sociological Science
Intersectionality theory argues that race and gender cannot be fully understood in isolation or through an additive approach. However, field experiments frequently examine aggregate racial or gender discrimination without accounting for within-category differences (e.g., gendered variations in racial discrimination) or intersectionality. Building on theoretical perspectives of intersectionality, we introduce a systematically comparative analytic framework designed to (a) identify missing results of correspondence audits and (b) provide recommendations to help scholars conduct more holistic analyses. We conduct a meta-analysis of 52 correspondence audits, encompassing nearly 330,000 tests for discrimination, and a re-examination of each study using seven discrimination ratios. The meta-analysis reveals a previously overlooked pattern in rental housing discrimination: compared to White men, Black men experience discrimination, Black women's outcomes are statistically similar, and White women receive preferential treatment. Additionally, our re-examination uncovers ten ways scholars may unintentionally miss key findings when failing to adopt a systematically comparative intersectional approach. We conclude with best-practice recommendations to guide scholars in designing, analyzing, and citing correspondence audits, helping them avoid these problematic scenarios. Importantly, our framework extends beyond field experiments focused on race and gender and can be broadly applied to research on disparities, enabling more comprehensive analyses across numerous dimensions.
- [34] Hans Hassell and John B. Holbein. "Navigating Potential Pitfalls in Difference-in-Differences Designs: Reconciling Conflicting Findings on Mass Shootings’ Effect on Electoral Outcomes" American Political Science Review 119, no. 1 (2025): 240-260.
Work on the electoral effects of gun violence in the U.S. relying on difference-in-differences designs has produced findings ranging from null to substantively large effects. However, as difference-in-difference designs, on which this research relies, have exploded in popularity, scholars have documented several methodological issues including potential violations of parallel-trends and unaccounted for treatment effect heterogeneity. These pitfalls (and their solutions) have not been fully explored in political science. We apply these advancements to the unresolved debate on gun violence's effects on U.S. electoral outcomes. We show that studies finding a large positive effect of gun violence on Democratic vote shares are a product of a failure to properly specify difference-in-differences models when underlying assumptions are unlikely to hold. Once these biases are corrected, shootings show little evidence of sparking large electoral change. Our work clarifies an unresolved debate and provides a cautionary guide for scholars currently employing difference-in-differences designs.
- [33] D'Wayne Bell, Jing Feng, John B. Holbein and Jonathan Smith. "Do STEM Students Vote?" American Educational Research Journal Vol. 61; no. 1 (2024): 48-73
Pundits, politicians, and academics have long worried about potentially low rates of civic participation among STEM-oriented students. Does studying STEM actually decrease the odds that young people will be actively involved in democracy? To answer this question, we created a dataset of over 23 million students in the United States, matched to national validated voting records. This novel dataset is the largest known individual-level dataset in the United States, connecting high school and college students to voting outcomes. It also contains a rich set of demographic and academic variables, to account for many of the common issues related to students' selection into STEM coursework. We consider two measures of STEM participation: Advanced Placement (AP) exam taking in high school and college major. Using both measures, we find that, across model specifications, the estimated relationships between STEM and voting are small in magnitude—about the same effect size as a single get-out-the-vote mailer. Our analyses demonstrate that, on average, marginally more STEM coursework in high school and college does not contribute noticeably to the low voting rates among young people in the United States.
- [32] Mia Costa, Charles Crabtree, John B. Holbein, and Michelangelo Landgrave. "Is that Ethical? An Exploration of Political Scientists’ Views on Research Ethics" Research & Politics 10, no. 4 (2023): 20531680231209553.
What do political scientists think about research ethics? What research practices do they find acceptable? Using a survey conducted with the American Political Science Association, we explore perceptions of ethics among 362 political scientists. We find that political scientists do not place much relative weight on ethics when evaluating research. We do, however, find that researchers view different modes of inquiry as having different potential harms. Furthermore, using a conjoint experiment, we find that factors like author affiliation, study location, method, and sample size shape evaluations of study ethicality. Our results contribute to the growing body of metascience by expanding understanding of ethics in political science research.
- [31] Tyler Reny, Ben Newman, John B. Holbein, and Hans Hassell. "Public mass shootings cause large surges in Americans' engagement with gun policy." PNAS nexus 2, no. 12 (2023): pgad407.
As public mass shootings continue to plague the United States, a growing scholarly literature seeks to understand the political effects of these tragic events. This literature, however, focuses on public opinion or turnout and vote choice, leaving open to question whether or not public mass shootings affect a range of other important actions citizens may take to engage with gun policy. Leveraging the as-goodas random timing of high-publicity public mass shootings over the past decade and an immense array of publicly available and proprietary data, we demonstrate that these events consistently cause surges in public engagement with gun policy—including internet searches, streaming documentaries, discussion on social media, signing petitions, and donating to political action committees. Importantly, we document the behaviors where shootings induce polarizing upswings in engagement and those where upswings skew toward gun control. Finally, we demonstrate that low-publicity shootings largely exert little-to-no effect on our outcomes.
- [30] Holbein, John B and Charles Crabtree. "Do Sleep Disruptions Promote Social Fragmentation?" Politics and the Life Sciences Vol. 42; no. 2 (2023): 205-233
Sleep changes predate shifts in mood/affect, thought processing, mental and physical health, civic engagement, and contextual circumstances, among other things. Theory predicts that these changes may lead to shifts in political and social beliefs. Do sleep disruptions shape how individuals see the world, the people around them, and themselves in relation to others? In this article, we use daily survey data from the 77 waves (N ≈ 460,000) of the University of California, Los Angeles’s 2019–2021 Nationscape Survey—a nationally representative political survey—to examine the effect of an exogenous short-term sleep disruption on measures of political views, polarization, and discriminatory beliefs. Using this data set, we leverage the modest sleep disruption that occurs at the start (and end) of Daylight Saving Time (DST) and employ a regression discontinuity in time design around the precise DST cutoff (which we supplement with event study models). Despite strong theoretical expectations and correlational connection between measures of sleep and many outcomes related to social fragmentation, we find that the DST change has little to no causal effect on citizens’ levels of polarization or their discriminatory attitudes. These effects are precise enough to rule out small effects, robust to a host of specification checks, and consistent across potential subgroups of interest. Our work adds to a small but growing body of research on the social and political effects of sleep disruptions.
- [29] Hillygus, D. Sunshine and John B. Holbein. "Refocusing Civic Education to Help Young People Develop the Skills They Need to Engage in Democracy" Annals of the American Academy of Political and Social Science Vol. 705; no. 1 (2023): 73-94
Schools have traditionally taken a "just-the-facts-ma'am" approach to civic education, focusing on governmental structures and political systems. We argue that preparing young people to engage with democracy requires far more than rote memorization of facts and figures. Schools should be laboratories of democracy, where young people's civic intentions are converted into civic behaviors. We argue that to realize that transformation, educators must impart real-world knowledge, practical skills, and nurturing abilities that are not captured by standardized tests of academic achievement: namely, the interpersonal and intrapersonal abilities conducive to civic mindedness. We discuss what these oft-labeled "noncognitive" skills are and how they are measured, review the evidence that shows how they foster democratic participation, articulate a vision for how civics can help develop students' noncognitive skills, and lay out a research agenda for scholars seeking to teach young people the skills requisite to actively participate in democracy.
- [28] Holbein, John B., Marcos Rangel, Raeal Moore, and Michelle Croft. “Is Voting Transformative? Expanding and Meta-Analyzing the Evidence” Political Behavior Vol. 45, no. 3 (2023): 1015-1044
Voting is the foundational act of democracy. While thousands of studies have treated voting as a dependent variable, comparatively little research has studied voting as an independent variable. Here we flip the causal arrow and explore the effect of exogenous voting shocks on citizens’ broader attitudes and behaviors. To do so, we first use two waves from a uniquely large survey of young people in the United States, pairing this with a regression discontinuity design. We augment these results with a new meta-analysis of all causally-identified studies exploring whether voting is transformative. We find that—despite voting at much higher rates—individuals induced to vote, regardless of the mode used to mobilize, are (precisely) no different from all-else-equal individuals that are not. Our results illuminate the (non)consequences of a vitally important—and widely studied—political behavior and speak to the broader importance of voting as an object of study.
- [27] Crabtree, Charles D., S. Michael Gaddis, Cameron Guage, John B. Holbein, Jae Yeon Kim, and William W. Marx. "Validated Names for Experimental Studies on Ethnicity and Race" Nature: Scientific Data. Vol. 10, no. 130 (2023) Pick names with this paper’s data →
A large and fast-growing number of studies across the social sciences use experiments to better understand the role of race in human interactions, particularly in the American context. Researchers often use names to signal the race of individuals portrayed in these experiments. However, those names might also signal other attributes, such as socioeconomic status (e.g., education and income) and citizenship. If they do, researchers would benefit greatly from pre-tested names with data on perceptions of these attributes; such data would permit researchers to draw correct inferences about the causal effect of race in their experiments. In this paper, we provide the largest dataset of validated name perceptions to date based on three different surveys conducted in the United States. In total, our data include over 44,170 name evaluations from 4,026 respondents for 600 names. In addition to respondent perceptions of race, income, education, and citizenship from names, our data also include respondent characteristics. Our data will be broadly helpful for researchers conducting experiments on the manifold ways in which race shapes American life.
- [26] Gaddis, S. Michael, Charles Crabtree, John B. Holbein, and Edvard Nergård Larsen. "Racially Distinctive Names Signal Race and Class" Sociological Science. Vol. 9 (2022): 454-472
Researchers studying discrimination and bias frequently conduct experiments that use racially distinctive names to signal race or ethnicity. The evidence that these studies provide about racial discrimination depends on the assumption that the names researchers use differ only based on perceived race and not some other factor. In this article, we assess this common assumption using data from five different studies (n = 1,004; 2,002; 1,035; 5,631; 1,858) conducted at different times across four separate survey platforms (Lucid Marketplace, Lucid Theorem, MTurk, and Prolific). We find evidence that names commonly used to signal race/ethnicity also influence perceptions about socioeconomic status and social class. Specifically, we observe that Americans tend to think that individuals with names typically used by Black and Hispanic people have lower educational attainment and income and are of a lower social class. Even when we present respondents with the educational attainment of a named individual, respondents still perceive Black people as lower social class than White people. We discuss the implications of these findings for past and future experimental work that uses names to signal race. We also articulate the importance of choosing names that best approximate the quantity that scholars want to estimate.
- [25] Block, Ray, Charles Crabtree, John B. Holbein, and Quin Monson. "Conceptual and Empirical Issues that Arise When Using Correspondence Audits to Measure Racial Discrimination." Proceedings of the National Academy of Sciences. Vol. 119, no. 38 (2022): e2210695119
Mitterer raises important issues in response to our work, but his conclusions are incorrect and incomplete. How we conceptualize and design experiments to test for discrimination matters a great deal. We agree that experiments using names as racial signals to study discrimination should consider what other characteristics beyond race people infer from names. We also agree that multiple names should be used; our work used five times the number of names commonly used, and these names were preregistered prior to data collection. Future work should continue to use multiple names but should do so with care. We disagree, however, with Mitterer's analytic approach. He analyzes our data as if it were collected with a between-subjects design when we actually used a within-subjects design. Mitterer ignores the considerable statistical-power gains that come from a within-subjects design. Mitterer's model with individual fixed effects—as is standard in within-subjects designs—shows robust evidence for discrimination, with effect sizes that are substantively and statistically indistinguishable from those provided in our original paper. We are ambivalent as to whether Mitterer's approach is best. We acknowledge model-based uncertainty. However, any analysis must correctly account for experimental design.
- [24] Mo, Cecilia Hyunjung, John B. Holbein, and Elizabeth Mitchell Elder. "Civilian National Service Programs Can Powerfully Increase Youth Voter Turnout." Proceedings of the National Academy of Sciences. Vol 119, no. 29 (2022): e2122996119
Low rates of youth voting are a feature of contemporary democracies the world over, with the United States having some of the lowest youth turnout rates in the world. However, far too little is known about how to address the dismal rates of youth voter participation found in many advanced democracies. In this paper, we examine the causal effect of a potentially scalable solution that has attracted renewed interest today: voluntary national service programs targeted at the youth civilian population. Leveraging the large pool of young people who apply each year to participate in the Teach For America (TFA) program—a prominent voluntary national service organization in the United States that integrates college graduates into teaching roles in low-income communities for 2 y—we examine the effect of service participation on voter turnout. To do so, we match TFA administrative records to large-scale nationwide voter files and employ a fuzzy regression discontinuity design around the recommended admittance cutoff for the TFA program. We find that serving as a teacher in the Teach For America national service program has a large effect on civic participation—substantially increasing voter turnout rates among applicants admitted to the program. This effect is noticeably larger than that of previous efforts to increase youth turnout. Our results suggest that civilian national service programs targeted at young people have great promise in helping to narrow the stubborn and enduring political engagement gap between younger and older citizens.
- [23] Barber, Michael, and John B. Holbein. "400 million voting records show profound racial and geographic disparities in voter turnout in the United States." PloS one 17, no. 6 (2022): e0268134.
One of the core tenets of a well-functioning representative democracy is that the people who vote to elect government officials are representative of the public. Here we reinforce the idea that reality is far from this lofty ideal. We document the extent and nature of inequities in voter participation in the United States with a level of granularity and precision that previous research has not afforded. To do so, we use a unique nationwide dataset of approximately 400 million validated voting records across multiple election cycles. With this novel dataset, we document large and persistent gaps in voter turnout by race, age, and political affiliation. Minority citizens, young people, and those who support the Democratic Party are much less likely to vote than whites, older citizens, and Republican Party supporters. Minorities, youth, and democrats are also much more likely to live in local communities where fewer individuals vote—areas that we term turnout deserts. Turnout deserts are especially pernicious given that they are self-reinforcing—bolstered by the social dynamics that fundamentally shape citizens' voting patterns. Our results show just how glaring inequities in political participation are in the US. These patterns threaten the very fabric of our democracy and fundamentally shift the balance of political power in the halls of government towards the interests of whites, older citizens, and republicans. They illustrate that participation in the United States is strikingly unequal—far from the ideals that this country has long aspired to.
- [22] Crabtree, Charles D., John B. Holbein, and Quin Monson. "Patients' Traits Shape Healthcare Providers' Choices on How to Best Allocate Life-Saving Care" Nature Human Behaviour 6 (2022): 244-257
During global pandemics, health-care decision makers often face critical shortages of life-saving medical equipment. How do medical stakeholders prioritize which patients are most deserving of scarce treatment? We report the results of three conjoint experiments conducted in the United States in 2020, testing for biases in US physicians’, citizens’ and elected politicians’ preferences for scarce ventilator distribution. We found that all stakeholders prioritized younger patients and patients who had a higher probability of surviving with ventilator access. When patients’ survivability was tied, physicians prioritized patients from racial/ethnic minorities (that is, Asian, Black and Hispanic patients) over all-else-equal white patients, religious minorities (that is, Muslim patients) over religious majority group members (that is, Catholic patients) and patients of lower socio-economic status over wealthier patients. The public also prioritized Black and Hispanic patients over white patients but were biased against religious minorities (that is, Atheist and Muslim patients) relative to religious majority group members. Elected politicians were also biased against Atheist patients. Our effects varied by political party—with Republican physicians, politicians and members of the public showing bias against religious minority patients and Democratic physicians showing preferential treatment of racial and religious minorities. Our results suggest that health-care stakeholders’ personal biases impact decisions on who deserves life-saving medical equipment.
- [21] Holbein, John B., Catherine Bradshaw, Nicholas Ialongo, B. Kal Munis, Jill Rabinowitz. "Promoting Voter Turnout: An Unanticipated Impact of Early Childhood Preventive Interventions." Prevention Science. Vol. 23 (2022): 192-203
Preventive interventions in early childhood have a range of behavioral and health effects. However, there is an emerging literature extending this work to include acts of civic engagement, such as voting. Given that America has one of the lowest and most disparate rates of voter turnout in the world—and most of the current efforts aimed at boosting voter turnout and making the electorate more representative of the general public are proximal to the voting experience—there is a need for a better understanding of the potential long-term impact of early-childhood programming on civic engagement in adulthood. This paper builds on theories of political socialization and prior research demonstrating significant impacts of the Fast Track preventive intervention on voter turnout to examine the extent to which there are positive impacts on voter participation for other evidence-based preventive interventions targeting children's social and emotional capacities. Specifically, we leveraged data from a randomized controlled trial of the classroom-centered (CC) and the family school partnership (FSP) preventive interventions delivered in first grade. We analyzed data from approximately 700 urban, predominately African American, public school students who were randomly assigned to classrooms that either implemented (1) the classroom-centered intervention (which included the good behavior game), (2) the FSP intervention, or (3) the business as usual (i.e., control group). Data from the trial were combined with archival voter data when the youth were in their early 30s. Analyses demonstrated positive impacts of the CC preventive intervention on voter turnout more than two decades after exposure to the prevention program. Taken together, these findings provide additional evidence that some of the attributes that promote active participation in democracy can be fostered in early childhood—long before most interventions that have previously tried, and often failed, to increase voter turnout.
- [20] Block, Ray, Charles Crabtree, John B. Holbein, and Quin Monson. "Are Americans Less Likely to Reply to Emails from Black People Relative to White People?" Proceedings of the National Academy of Sciences. Vol. 118, no. 52 (2021): e2110347118
In this article, we present the results from a large-scale field experiment designed to measure racial discrimination among the American public. We conducted an audit study on the general public—sending correspondence to 250,000 citizens randomly drawn from public voter registration lists. Our within-subjects experimental design tested the public's responsiveness to electronically delivered requests to volunteer their time to help with completing a simple task—taking a survey. We randomized whether the request came from either an ostensibly Black or an ostensibly White sender. We provide evidence that in electronic interactions, on average, the public is less likely to respond to emails from people they believe to be Black (rather than White). Our results give us a snapshot of a subtle form of racial bias that is systemic in the United States. What we term everyday or "paper cut" discrimination is exhibited by all racial/ethnic subgroups—outside of Black people themselves—and is present in all geographic regions in the United States. We benchmark paper cut discrimination among the public to estimates of discrimination among various groups of social elites. We show that discrimination among the public occurs more frequently than discrimination observed among elected officials and discrimination in higher education and the medical sector but simultaneously, less frequently than discrimination in housing and employment contexts. Our results provide a window into the discrimination that Black people in the United States face in day-to-day interactions with their fellow citizens.
- [19] Pfaff, Steven, Charles Crabtree, Holger L. Kern, and John B. Holbein. "Do Street‐Level Bureaucrats Discriminate Based on Religion? A Large‐Scale Correspondence Experiment among American Public School Principals" Public Administration Review. Vol. 81, no. 2 (2021): 244-259
Although public administration scholars have long studied discrimination on the basis of race/ethnicity, class, and gender, little to no research exists on whether street-level bureaucrats provide differential services based on the religious identity of their constituents. This article reports the results from a large-scale correspondence study of street-level bureaucrats in the American public school system. The authors emailed the principals of a large sample of public schools and asked for a meeting, randomly assigning the religious (non)affiliation of the family. To get at potential causal mechanisms, religious belief intensity was also randomly assigned. The findings show evidence of substantial discrimination against Muslims and atheists on a par with, and sometimes larger than, the racial discrimination found in previous studies. These individuals are substantially less likely to receive a response, with discrimination growing when they signal that their beliefs are more intense. Protestants and Catholics face no discrimination unless they signal that their religious beliefs are intense.
- [18] Easton, Matt and John B. Holbein. "The Democracy of Dating: How Political Affiliations Shape Relationship Formation." Journal of Experimental Political Science. Vol. 8, no. 3 (2021): 260-272.
How much does politics affect relationship building? Previous experimental studies have come to vastly different conclusions – ranging from null to truly transformative effects. To explore these differences, this study replicates and extends previous research by conducting five survey experiments meant to expand our understanding of how politics does/does not shape the formation of romantic relationships. We find that people, indeed, are influenced by the politics of prospective partners; respondents evaluate those in the political outgroup as being less attractive, less dateable, and less worthy of matchmaking efforts. However, these effects are modest in size – falling almost exactly in between previous study estimates. Our results shine light on a literature that has, up until this point, produced a chasm in study results – a vital task given concerns over growing levels of partisan animus in the USA and the rapidly expanding body of research on affective polarization.
- [17] Barber, Michael and John B. Holbein. "The Participatory and Partisan Impacts of Mandatory Vote-by-Mail" Science Advances Vol. 6, no. 35 (2020): eabc7685
Recently, mandatory vote-by-mail has received a great deal of attention as a means of administering elections in the United States. However, policy-makers disagree on the merits of this approach. Many of these debates hinge on whether mandatory vote-by-mail advantages one political party over the other. Using a unique pairing of historical county-level data that covers the past three decades and more than 40 million voting records from the two states that have conducted a staggered rollout of mandatory vote-by-mail (Washington and Utah), we use several methods for causal inference to show that mandatory vote-by-mail slightly increases voter turnout but has no effect on election outcomes at various levels of government. Our results find meaning given contemporary debates about the merits of mandatory vote-by-mail. Mandatory vote-by-mail ensures that citizens are given a safe means of casting their ballot while simultaneously not advantaging one political party over the other.
- [16] Hassell, Hans, John B. Holbein, and Matthew Baldwin. "Mobilize for Our Lives? School Shootings and Democratic Accountability in U.S. Elections" American Political Science Review. Vol. 114, no. 4 (2020): 1375-1385
Gun violence is a large and growing problem in the United States. Many reformers look towards elections to spur policy change in this area. In this paper, we explore the effects of school shootings on electoral mobilization and election outcomes. We pair data from several sources that measure validated voter registration; validated voter turnout; and the electoral performance of officials at the local, state, and federal levels with regression discontinuity and panel methods. Our effects show that shootings have little to no effect on electoral outcomes in the United States. Our work demonstrates that even when tragic events occur that are squarely in the realm of elected officials' responsibility, have high levels of issue salience, are highly-covered by the media, draw citizens' attention, and (perhaps) shift public opinion, these seemingly favorable conditions may not be enough to elicit democratic accountability.
- [15] Holbein, John B. and Marcos Rangel. "Does Voting Have Upstream and Downstream Consequences? Regression Discontinuity Tests of the Transformative Voting Hypothesis." The Journal of Politics. Vol. 82, no. 4 (2020): 1196-1216
Voting is a central pillar of political science research; indeed, scholars have long addressed questions like, “Who votes?,” “Why do people vote?,” and “What interventions increase voting?” However, only a few have considered whether voting changes adjacent civic dispositions and behaviors. In this paper, we explore the effects of voting shocks on young citizens’ political interest, memberships, social awareness, and political knowledge in the lead up to the voting experience (“upstream”) and in the months and years after (“downstream”). To do so, we use a unique combination of large survey data from two countries paired with an exact date-of-birth regression discontinuity design. We find that eligibility to vote voluntarily and exposure to compulsory voting—despite eliciting large turnout increases—have precisely estimated null effects on young people’s upstream or downstream civic engagement. While voting may be an important experience, it appears to have smaller transformative effects than previously thought.
- [14] Holbein, John B., D. Sunshine Hillygus, Matthew A. Lenard, Christina Gibson-Davis, Darryl V. Hill, and. "The Development of Students’ Engagement in School, Community, and Democracy" British Journal of Political Science. Vol. 50, no. 4 (2020): 1439-1457
This article explores the origins of youth engagement in school, community and democracy. Specifically, it considers the role of psychosocial or non-cognitive abilities, like grit or perseverance. Using a novel original large-scale longitudinal survey of students linked to school administrative records and a variety of modeling techniques – including sibling, twin and individual fixed effects – the study finds that psychosocial abilities are a strong predictor of youth civic engagement. Gritty students miss less class time and are more engaged in their schools, are more politically efficacious, are more likely to intend to vote when they become eligible, and volunteer more. Our work highlights the value of psychosocial attributes in the political socialization of young people.
- [13] Hassell, Hans, John B. Holbein, and Matt Miles. "There Is No Liberal Media Bias in Which News Stories Political Journalists Choose to Cover" Science Advances Vol. 6, no. 14 (2020): eaay9344
Is the media biased against conservatives? Although a dominant majority of journalists identify as liberals/Democrats and many Americans and public officials frequently decry supposedly high and increasing levels of media bias, little compelling evidence exists as to (i) the ideological or partisan leanings of the many journalists who fail to answer surveys and/or identify as independents and (ii) whether journalists' political leanings bleed into the choice of which stories to cover that Americans ultimately consume. Using a unique combination of a large-scale survey of political journalists, data from journalists' Twitter networks, election returns, a large-scale correspondence experiment, and a conjoint survey experiment, we show definitively that the media exhibits no bias against conservatives (or liberals for that matter) in what news that they choose to cover. This shows that journalists' individual ideological leanings have unexpectedly little effect on the vitally important, but, up to this point, unexplored, early stage of political news generation.
- [12] Akee, Randall, William Copeland, John B. Holbein, and Emilia Simeonova. "Human Capital and Voting Behavior across Generations: Evidence from an Income Intervention." American Political Science Review 114, no. 2 (2020): 609-616
Despite clear evidence of a sharp income gradient in voting participation, it remains unclear whether income truly causes voting. In this article, we investigate how exogenous increases in unearned income affect voting in U.S. elections for two generations (parents and children) from the same household. In contrast to predictions made by current models of voting, we find the income shock had no effect on parents' voting behaviors. However, we also find that increasing household income has heterogeneous effects on the civic participation of children from different socioeconomic backgrounds. It increases children's voting propensity among those raised in initially poorer families—resulting in substantially narrowed participatory gaps. Our results are consistent with a more nuanced view of how individual resources affect patterns of voting than the dominant theoretical framework of voting—the resource model—allows. Voting is fundamentally shaped by the human capital accrued long before citizens are eligible to vote.
- [11] Dynes, Adam and John B. Holbein. "Noisy Retrospection: The Effect of Party Control on Policy Outcomes" American Political Science Review 114, no. 1 (2020): 237-257.
Retrospective voting is vital for democracy. But, are the objective performance metrics widely thought to be relevant for retrospection—such as the performance of the economy, criminal justice system, and schools, to name a few—valid criteria for evaluating government performance? That is, do political coalitions actually have the power to influence the performance metrics used for retrospection on the timeline introduced by elections? Using difference-in-difference and regression discontinuity techniques, we find that US states governed by Democrats and those by Republicans perform equally well on economic, education, crime, family, social, environmental, and health outcomes on the timeline introduced by elections (2–4 years downstream). Our results suggest that voters may struggle to truly hold government coalitions accountable, as objective performance metrics appear to be largely out of the immediate control of political coalitions.
- [10] Schafer, Jerome and John B. Holbein. "When Time Is of the Essence: A Natural Experiment on How Time Constraints Influence Elections" The Journal of Politics 82, no. 2 (2020): 418-432
Foundational theories of voter turnout suggest that time is a key input in the voting decision, but we possess little causal evidence about how this resource affects electoral behavior. In this article, we use over two decades of elections data and a novel geographic regression discontinuity design that leverages US time zone boundaries. Our results show that exogenous shifts in time allocations have significant political consequences. Namely, we find that citizens are less likely to vote if they live on the eastern side of a time zone border. Time zones also exacerbate participatory inequality and push election results toward Republicans. Exploring potential mechanisms, we find suggestive evidence that these effects are the consequence of insufficient sleep and moderated by the convenience of voting. Regardless of the exact mechanisms, our results indicate that local differences in daily schedules affect how difficult it is to vote and shape the composition of the electorate.
- [9] Fraga, Bernard and John B. Holbein. "Measuring Youth and College Student Voter Turnout." Electoral Studies 65, no. 3 (2020).
Young adulthood is a critical period in civic development. However, measuring electoral participation among this group generally—and the many young people who go to college in particular—is fraught with potential pitfalls stemming from a reliance on survey-based measures of voting. In this note, we compare patterns of youth turnout in two large-scale, survey-based datasets commonly used to measure voting, the Current Population Survey and the Cooperative Congressional Election Study, to two voter-file based datasets: the National Study of Learning, Voting, and Engagement (NSLVE) and a comprehensive nationwide voterfile provided by the Data Trust. We find high levels of concordance between measures in the NSLVE, Data Trust, and the CPS. However, despite linking their sample to validated voter records, the CCES does not mirror these benchmarks. We conclude by discussing the challenges and opportunities that shape the study of youth turnout.
- [8] Holbein, John B., Jerome Schafer, and David Dickinson. "Insufficient Sleep Reduces Voting and Other Prosocial Behaviors." Nature Human Behaviour 3, May (2019): 492-500. [Impact Factor: 24.25]
Insufficient sleep is a growing public health concern in industrial societies. Although a lack of sleep is known to negatively affect private behaviours—such as working or going to school—comparatively little is known about its consequences for the social behaviours that hold society and democracy together. Using three complementary methods, we show how insufficient sleep affects various measures of civic participation. With survey data from two countries, we show that insufficient sleep predicts lower voter turnout. Next, with a geographical regression discontinuity design, we demonstrate that individuals from the United States who tend to sleep less due to circadian impacts of time-zone boundaries are also less likely to vote. Finally, we experimentally manipulate short-term sleep over a two-stage study. We observe that the treatment decreases the levels of civic engagement, as shown by their willingness to vote, sign petitions and donate to charities. These results highlight the strong negative consequences that current levels of insufficient sleep have on vitally important measures of social capital.
- [7] Carnes, Nicholas and John B. Holbein. "Do Public Officials Exhibit Social Class Biases when they Handle Casework? Evidence from Multiple Correspondence Experiments" PLOS ONE 14, no. 3 (2019).
Are public officials more responsive to requests from affluent or poor constituents? A growing body of evidence suggests that lawmakers are more responsive to the rich when they craft policy. However, some scholars theorize that officials also exhibit a corresponding bias in favor of the poor when they handle casework, essentially giving policy to the rich and services to the poor. In this paper, we test this casework prediction using four experiments in which confederates sent simple requests to state or local officials. In each, our confederates' reported social classes were randomly assigned and signaled with a brief introductory statement mentioning the sender's occupation or economic situation. Across our samples, we find precisely-estimated null effects of social class biases: the officials we studied were equally likely to respond regardless of the constituent's class. These findings raise doubts about whether casework is really a class-biased process.
- [6] Holbein, John B. and Hans J.G. Hassell. "When Your Group Fails: The Effect of Race-Based Performance Signals on Citizen Voice and Exit" Journal of Public Administration Research and Theory 29, no. 2 (2019): 268–286.
In recent years, performance-based accountability reforms have become widespread—particularly in the public education system—as a means of enhancing service delivery and citizens' knowledge of and engagement with government. While an ever-growing literature has considered the overall effects of these administrative systems, few have fully considered the effect these have on underlying subgroups of relevance and inequalities in responses across these groups. In this article, we examine how citizens of various racial subgroups respond when governments send negative race-based signals about the performance of inand out-racial groups. Specifically, we explore whether No Child Left Behind's (NCLB) race-based failure signals affect racial groups' use of voice in school board elections and their use of exit from local public schools. To do so, we combine school and voter administrative records in North Carolina with a regression discontinuity design that leverages exogenous variation around NCLB's race-based school failure cutoffs. Consistent with our theoretical framework, we find that white and black citizens respond differently to race-based failure signals. We find the whites are more responsive overall: responding to failure signals of various types both by voting in local school board elections and exiting local failing schools. African Americans, however, seem much more sensitive to racialized cues—responding at the ballot box but not by exiting when their group fails. These results show that while performance accountability systems have broader effects than previously realized: shifting the composition of local elections and distorting the racial makeup of schools.
- [5] Holbein, John B. “Childhood Skill Development and Adult Political Participation.” American Political Science Review 111, no. 3 (2017): 572-583.
Recent child development research shows that the psychosocial or noncognitive skills that children develop—including the ability to self-regulate and integrate in social settings—are important for success in school and beyond. Are these skills learned in childhood also important for adult political behaviors like voting? In this article, I use a unique school-based 20-year field experiment to explore whether children who develop psychosocial skills early on are more likely to vote in adulthood than those who do not. Matching subjects to voter files, I show that this intervention had a noticeable long-run impact on political participation. These results highlight the need to better understand how childhood experiences shape civic behaviors later in life. During this critical period, children can be taught the not explicitly political, but still vital, skills that set them on a path toward political participation in adulthood.
- [4] Holbein, John B. & Helen F. Ladd. “Accountability pressure: Regression discontinuity estimates of how No Child Left Behind influenced student behavior.” Economics of Education Review 58 (2017): 55-67
In this paper we examine how failing to make adequate yearly progress under No Child Left Behind (NCLB), and the accountability pressure that ensues, affects various non-achievement student behaviors. Using administrative data from North Carolina and leveraging a discontinuity in the determination of school failure, we examine the causal impact of this form of accountability pressure both on student behaviors that are incentivized by NCLB and on those that are not. We find evidence that, as NCLB intends, pressure encourages students to show up at school and to do so on time. Accountability pressure also appears to have the unintended effect, however, of increasing the number of student misbehaviors. Further, we find some evidence that this negative response is most pronounced among minorities and low performing students: those who are the most likely to be left behind.
- [3] Ladd, Helen F., Charles T. Clotfelter, and John B. Holbein. “The Growing Segmentation of the Charter School Market in North Carolina” Education Finance and Policy 12, no. 4 (2017): 536-563.
A defining characteristic of charter schools is that they introduce a strong market element into public education. In this paper, we examine through the lens of a market model the evolution of the charter school sector in North Carolina between 1999 and 2012. We examine trends in the mix of students enrolled in charter schools, the racial imbalance of charter schools, patterns in student match quality by schools' racial composition, and the distributions of test score performance gains compared to those in traditional public schools. In addition, we use student fixed effects models to examine plausibly causal measures of charter school effectiveness. Our findings indicate that charter schools in North Carolina are increasingly serving the interests of relatively able white students in racially imbalanced schools and that despite improvements in the charter school sector over time, charter schools are still no more effective on average than traditional public schools.
- [2] Holbein, John B. “Left Behind? Citizen Responsiveness to Government Performance Information.” American Political Science Review 110, no. 2 (2016): 353-368.
Do citizens respond to policy-based information signals about government performance? Using multiple big datasets—which link for the first time large-scale school administrative records and individual validated voting behavior—I show that citizens react to exogenous school failure signals provided by No Child Left Behind. These signals cause a noticeable increase in turnout in local school board elections and increase the competitiveness of these races. Additionally, I present evidence that school failure signals cause citizens to vote with their feet by exiting failing schools, suggesting that exit plays an underexplored role in democratic accountability. However, performance signals elicit a response unequally, with failure primarily mobilizing high propensity citizens and encouraging exit among those who are white, affluent, and more likely to vote. Hence, while performance signals spur a response, they do so only for a select few, leaving many others behind.
- [1] Holbein, John B. and D. Sunshine Hillygus. "Making Young Voters: The Impact of Preregistration on Youth Turnout." American Journal of Political Science 60, no. 2 (2016): 364-382 [Correction] [Additional analysis].
Recent research has cast doubt on the potential for various electoral reforms to increase voter turnout. In this article, we examine the effectiveness of preregistration laws, which allow young citizens to register before being eligible to vote. We use two empirical approaches to evaluate the impact of preregistration on youth turnout. First, we implement difference-in-difference and lag models to bracket the causal effect of preregistration implementation using the 2000–2012 Current Population Survey. Second, focusing on the state of Florida, we leverage a discontinuity based on date of birth to estimate the effect of increased preregistration exposure on the turnout of young registrants. In both approaches, we find preregistration increases voter turnout, with equal effectiveness for various subgroups in the electorate. More broadly, observed patterns suggest that campaign context and supporting institutions may help to determine when and if electoral reforms are effective.
Revisions Requested
- [22] D'Wayne Bell, John B. Holbein, Sam Imlay, and Jonathan Smith. "Which Colleges Increase Voting Rates?" Revised and Resubmitted at the American Economic Journal: Applied Economics
Using millions of SAT takers linked to college enrollment and voting records, we document wide variation in voter participation across otherwise similar colleges. We estimate over 2,600 colleges' value-added to the likelihood of voting and find that 30 percent of the observed variation across colleges is attributable to the colleges themselves. The standard deviation of these estimates ranges from 3 to 5 percentage points (6–10 percent), with almost all students in our sample attending colleges that increase their entrants' probability of voting. These effects appear soon after enrollment, persist beyond college, and are driven by colleges creating new voters.
- [21] Michael Bloem, John B. Holbein, Sam Imlay, and Jonathan Smith. “Voting Among Siblings” Invitation to Revise and Resubmit at American Economic Journal: Economic Policy
Using millions of siblings in the U.S., we detail three findings that quantify whether siblings influence one another to vote in national elections. First, and descriptively, younger siblings are 10 percentage points (50 percent) more likely to vote in their first eligible election when their older sibling votes in a prior election. Second, roughly one-third of this is caused by the older sibling voting, as determined by age-of-voting-eligibility thresholds in a regression discontinuity design. Third, the causal impact of a sibling voting runs in the other direction as well—younger siblings increase the probability of their older siblings voting in their early 20’s by 14 percent. These results demonstrate the influence and importance of family and peers in creating an engaged citizenry and underscore that across a wide array of policy domains, conventional impact evaluations do not fully account for all of policies’ impacts.
- [20] Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and J. Quin Monson. “Micro-Inequities Against Asian Americans: Evidence from a Large-Scale Field Experiment” Revised and Resubmitted at the American Political Science Review
To what extent do Asians face micro-inequities, and can signaling cultural assimilation reduce this bias? We examine these questions through a field experiment involving 250,000 randomly sampled registered voters in the United States. Participants received emails with a simple request for help from senders whose randomized racial identity (Asian or white) was signaled through their names. Among senders with putatively Asian names, we randomized whether the first name was Western or anglicized — a practice often known as "name whitening." We find that micro-inequities against Asians are (1) large, precisely estimated, and robust; (2) systemic across the U.S.; (3) common across most demographic subgroups; and (4) uncorrelated with previously documented overt forms of anti-Asian bias. Leveraging both the field experiment and original survey data on how names are perceived, we test nine potential mechanisms: perceptions of Asian identity, perceived foreignness, perceived socioeconomic status, perceived character flaws, perceptions of name inauthenticity toward Asian-origin names (i.e., the inclination to regard such names as artificial or automated ("bot") identities rather than real individuals), filtering by spam detection algorithms, a lack of potential exposure to Asians, high affectedness to COVID-19, and high economic resentment toward China. Our results are consistent with discrimination against Asians being driven, in large part, by perceptions of foreignness or non-citizenship and, secondarily, the misperception that Asian-origin names are not associated with real people. While anglicizing Asian names reduces the discrimination we detected by about half, it does not eliminate it. These results provide some of the clearest evidence to date on the nature and causes of micro-inequities faced by Asians in the United States.
- [19] Charles Crabtree, Will Elliott, John B. Holbein, Devontae Lacasse, and Jayanth Uppaluri. "What Does the Public Really Want in College Admissions?" Invitation to Revise and Resubmit at Nature Human Behaviour
What types of students should be admitted to prestigious colleges according to the American public? We conducted a large conjoint experiment with a sample of 2,580 U.S. adults; respondents evaluated a total of 25,800 hypothetical applicants. We randomly varied the applicants’ race, academic credentials, socioeconomic background, country of origin, and disability status. We find that academic credentials dominate public preferences: the public strongly prefers college applicants with higher GPAs, stronger SAT scores, and/or evidence of leadership in athletics or extracurricular activities. By contrast, race has only modest effects; in their actions within the conjoint, citizens indicate a preference that racial groups be treated comparatively similarly in admissions decisions. Though there are some differences by respondents’ own race/ethnicity—with Black and Hispanic individuals favoring their groups being, all-else-equal, given preferential treatment—these differences are small relative to other features. In contrast, there is some evidence that the public believes that college admissions should take into account the difficulties faced by socioeconomically disadvantaged individuals—indeed, our experimental results show that the public prefers low-income applicants to receive sizable advantages over high-income applicants. Simultaneously, however, the public prefers that students from the United States receive preferential admissions over immigrants. Finally, we show that the largest penalties arise from mental health disclosure: psychiatric conditions reduce applicants’ selection probability by a substantial amount.
- [18] Charles Crabtree, S. Michael Gaddis, and John B. Holbein. "How Should Researchers Signal Race in Correspondence Audits? Considering The Validity of Names and Photographs" Rejection with an Invitation to Resubmit at the Journal of Politics
How should researchers signal race in correspondence audits of discrimination? In a forthcoming paper at the Journal of Politics, Kaufman, Celaya, and Grumbach (hereafter KCG) argue that researchers should use photos rather than names to signal race. They claim that name-based treatments suffer from “treatment non-compliance” and propose using photos as a solution. However, we show this claim is neither novel nor their solution effective. Using a national, quota-based survey of 1,400 U.S. adults, we demonstrate that respondents frequently perceive emails with photos as spam, that senders rarely include photos in many real-world communication settings, and that the images used by KCG send bundled and unpredictable signals beyond race. Together with our analytical critique, this empirical evidence suggests KCG’s approach lowers ecological validity and signal clarity and does not improve signal strength. We recommend caution and propose alternative improvements for experimental discrimination research.
- [17] Michael Gaddis, Charles Crabtree, John B. Holbein, and Steve Pfaff. "Racial/Ethnic Discrimination in Interactions with the U.S. Public Education System: A Correspondence Audit of Principals." Invitation to Revise and Resubmit at Sociology of Education
Although numerous studies document different forms of discrimination in the U.S. public education system, very few provide plausibly causal estimates. Thus, it is unclear to what extent public school principals discriminate against racial and ethnic minorities. Moreover, no studies test for heterogeneity in racial/ethnic discrimination by individual-level resource needs and school-level resource strain – potentially important moderators in the education context. Using a correspondence audit, we examine bias against Black, Hispanic, and Chinese American families in interactions with 52,792 public K-12 principals in 33 states. Our research provides causal evidence that Hispanic and Chinese American families face significant discrimination in initial interactions with principals, regardless of individual-level resource needs. Black families, however, only face discrimination when they have high resource needs. Additionally, principals in schools with greater resource strain discriminate more against Chinese American families. This research uncovers complexities of racial/ethnic discrimination in the K-12 context because we examine multiple racial/ethnic groups and test for heterogeneity across individualand school-level variables. These findings highlight the need for researchers conducting future correspondence audits to expand the scope of their research to provide a more comprehensive analysis of racial/ethnic discrimination in the U.S.
- [16] Michael Gaddis, Charles Crabtree, John B. Holbein, and Steve Pfaff. "Local Political Party Voting Context Moderates Public School Principals’ Levels of Racial/Ethnic Discrimination" Invitation to Revise and Resubmit at the Journal of Race, Ethnicity, and Politics
Correspondence audits document causal evidence of racial/ethnic discrimination in many contexts. However, few studies have examined whether local political party voting context influences individuals to engage in “stakeholder-centric” discrimination on behalf of or in response to expectations of others. We examine heterogeneity in racial/ethnic discrimination by the county-level Republican vote share with a correspondence audit of 52,792 K-12 public-school principals across 33 states. We email principals posing as parents considering a move to the school district and requesting a meeting. We find evidence that the county-level Republican vote share strongly moderates racial/ethnic discrimination against Black and Chinese American families. While all groups are less likely to receive a response from principals as the Republican vote share increases, the declines for Black and Chinese American families are largest. Thus, discrimination against Black and Chinese American families is sizable in counties with the highest Republican vote share. These findings shed light on how partisanship can shape the experiences of historically marginalized groups. Furthermore, there may be benefits to targeting limited resources to geographies where discrimination is more likely to occur.
- [15] Charles Crabtree, John B. Holbein, and Kiyoteru Tsutsui. "The Tyranny of Alphabetical Ordering: The Uneven Distribution of Authorship Credits by Authors' Country of Origin" Revised and Resubmitted at the Journal of Informetrics
Many scholars treat alphabetical ordering of authorship as a neutral convention for allocating credit in academic publishing in the social sciences. We show that this practice systematically advantages scholars from some countries while disadvantaging others, generating a hidden and persistent source of global inequality in academic recognition. Drawing on a century of publications from leading journals in economics, political science, and sociology, we document that scholarly surnames cluster around a median initial of K, creating a structural threshold that determines alphabetical advantage versus disadvantage under alphabetized bylines. We then combine these publication patterns with the most comprehensive global database of name usage to date — nearly half a billion name profiles across more than 100 countries — to characterize country-level surname-initial distributions. Our analysis reveals stark cross-national disparities: scholars from countries whose population surnames cluster later in the alphabet systematically appear later in bylines. These disadvantages cluster geographically, falling disproportionately on South America and East and Central Asian countries. Regression analyses suggest that cultural-linguistic factors, particularly script families, rather than economic development, better explain cross-country variation in surname-initial distributions. Our findings demonstrate that alphabetical ordering operates not as a neutral convention but as a structural mechanism that reproduces global inequalities in academic credit.
- [14] Charles Crabtree and John B. Holbein. "Attention-Check Failure as a Survey-Level Diagnostic of Human Participation in Online Surveys" Rejection with an Invitation to Resubmit at Proceedings of the National Academy of Sciences
AI agents now pass attention checks at near-perfect rates, but the field’s most pressing data quality challenge is not AI contamination—it is the enormous variation in human respondent quality across survey platforms and the absence of any public data with which to benchmark it. We propose that providers publish aggregate attention-check passage rates at regular intervals. To show why, we formalize a mixture-model diagnostic exploiting the asymmetry that humans fail attention checks at nontrivial rates while AI agents do not, and apply it to published cross-platform data. The diagnostic fails as a cross-platform tool because human quality variation overwhelms any contamination signal; it succeeds only as a within-platform longitudinal monitor—which requires baselines no provider currently publishes. A correspondence with twelve survey providers confirms this gap. We call on providers to make baseline data public and on journals to require their disclosure.
Additional Works Under Review
- [13] Hans Hassell, Taylor Hill, and John B. Holbein. "Do ICE Arrests Spark Electoral Action?" Under Review
Does U.S. Immigration and Customs Enforcement (ICE) activity trigger electoral mobilization? Theory and prior evidence suggest it should, as deportation imposes exceptionally high stakes on affected communities. Indeed, contemporary enforcement has sparked widespread protests. But do these actions translate into electoral engagement? The second Trump administration’s enforcement surge represents the most favorable conditions ever observed for detecting an electorally focused response. Using a nationwide panel from 2023 to 2025, we test whether ICE activity has translated into changes in voter registration and political donations via difference-in-differences, event study, and regression discontinuity designs. We find largely null or substantively small estimates. We can rule out even very modest changes in voter registration patterns overall and across subgroups, while donation effects are substantively negligible and offsetting across parties. Results survive a demanding critical-case test using Minnesota ICE activity in late 2025. Even historically unprecedented enforcement does not automatically translate into political mobilization.
- [12] Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and Surili Sheth. "Do Virtual Museums Highlighting the Experiences of Minorities Persuade Visitors? Evidence From a Study on Bias Toward Asian Americans" Under Review
Biases against individuals of Asian descent persist in the United States. Can cultural history museums, which aim to provide public education on the struggles, achievements, and cultural traditions of different groups, reduce these biases? We address this question through an experiment using virtual museum content curated by the Museum of Chinese in America (MOCA). By embedding MOCA’s virtual exhibits into surveys, we test whether exposure to the lived experiences and history of the Chinese diaspora in the United States alters non-Asian Americans’ attitudes toward and knowledge about Asian Americans. Our findings suggest that historical and emotionally resonant messaging common in museum exhibits improve attitudes and beliefs, primarily among individuals who already hold positive racial predispositions. These results highlight both the promise and limits of museum-based interventions aimed at reducing out-group bias, and speak to the efficacy of scalable digital interventions.
- [11] Charles Crabtree, Michael Gaddis, and John B. Holbein. "Shifting Paradigms: Nine Decades of Identity and Discrimination Scholarship in Academic Research" Under Review
How has academic attention to identity and discrimination evolved over time? Analyzing 216,500 English-language research articles from JSTOR (1933–2025), we document temporal trends in twelve identity categories: race, ethnicity, gender, sex, class, religion, immigration, disability, sexuality, indigenous peoples, minority status, and discrimination. Three major transformations emerge. First, a "gender turn" beginning in the 1970s saw gender keywords rise from 1% to 11% by 2020, while sex-based framing declined from 40% (1980s peak) to 25% today. Second, attention to class peaked at 29% during the 1970s economic crises but declined through the neoliberal era to 17% by 2020. Third, immigration scholarship surged from 6% to 12% (2000–2020). We also document emerging categories: disability (1% to 3%), sexuality (0% to 3% since 1990), and indigenous peoples (3% to 8%). Secondary analysis of 20,706 race/ethnicity articles reveals that "White" keywords mark Critical Whiteness Studies (90.5% co-occur with "race"), not reverse discrimination claims, while Asian American scholarship reflects post-1965 immigration and model minority debates. Together, these patterns provide a systematic longitudinal account of how identity categories enter, expand within, and recede from the scholarly agenda.
- [10] Hans Hassell, John B. Holbein, Kevin Reuning, and Cindy West. "The Electoral Consequences of Social Protest in Contemporary America" Under Review
Do protest-led social movements change elections? Virtually all prior research says yes, and markedly so. However, this research has often struggled to identify causal effects or, even when it has, tended to examine singular, high-salience movements, leaving open what happens when protest is a sustained, system-wide phenomenon. We address this gap by focusing on the United States and merging protest event data from the Crowd Counting Consortium, which captures nearly 200,000 events from small local gatherings to mass mobilizations across every ideological stripe, with nationwide voter files, turnout and election returns, and FEC donation records from 2017 through 2024. Using difference-in-differences and event study techniques, we find effects that are, with few exceptions, small at best and broadly indicative of a null effect. This null holds across protests that span the ideological spectrum, the target and size of the protest, and is consistent across demographic subgroups. The limited exceptions are revealing: in the rare instances where confrontational protests occur—resulting in injuries or property destruction and comprising no more than ≈ 1% of all protest events—we observe a modest bipartisan donation surge to both Democratic and Republican candidates, with no consistent net advantage for either party. In subsequent elections, we also observe a small and somewhat sensitive to model specification increase in Democratic vote share. Overall, however, in a deeply polarized era, we find that protest effects on elections are muted.
Additional Working Papers
- [9] John B. Holbein. "Do Natural Disasters Really Move Votes? Why the Evidence Does Not Survive Modern Difference-in-Differences Methods"
Natural disasters are a central proving ground for retrospective voting: if voters hold governments to account, disastrous events should show it. The evidence rests on a few foundational studies estimated before difference-in-differences research confronted the fragility of its core assumption, parallel trends. I revisit the two most influential—Gasper and Reeves (2011) and Healy and Malhotra (2009)—on their own data. Both reproduce. In both, however, pre-treatment trends do not run parallel: the outcomes of treated counties were already trending before any disaster arrived, and the headline effects—the disaster's and the response's alike—do not survive modest, formal allowances for those violations. Extending the test across a half-century of elections with damage and response measures yields the same null at the federal, state, and local levels. Voters may respond to disasters far more tepidly, if at all, than previously thought.
- [8] John B. Holbein. "How Underpowered Are Close-Election Regression Discontinuity Designs? Evidence from U.S. Senate, House, Gubernatorial, State Legislative, and Local Races"
Close-election regression discontinuity (RD) designs are widely used given their promise of clean identification of the electoral effects of holding office. While facing scrutiny previously surrounding identification issues and the relevance of the estimands they produce, their statistical power has gone largely unexamined. Using contested U.S. Senate, House, gubernatorial, state legislative, and local election margins, this paper documents how severely underpowered close election regression discontinuity designs typically are. At the ±5-percentage-point bandwidth common in published work, Senate and gubernatorial designs reach under 10% power for a small effect (Cohen's d = 0.2) and the House 27%; even at ±10 pp the House reaches just 48%. In Congress the close races never accumulate fast enough: reaching 80% power would take the House over 150 years and the Senate more than a millennium. Pooled subnational designs are the exception, clearing conventional power across thousands of state legislative and local races; but that power belongs to the pool, and a single state or city, even over decades, falls back to single digits. Underpowered designs that do reach significance badly exaggerate effects and occasionally reverse their sign. A re-analysis of a prominent extremist-primaries RD makes the stakes concrete: its minimum detectable effect exceeds the largest effect the study reports. Scholars should exercise caution when leveraging this research design.
- [7] Charles Crabtree, John B. Holbein, and Cecilia Hyunjung Mo. "Cultural Assimilation Weakens Workers’ Discriminatory Preferences Against Potential Asian Bosses"
We document that a large sample of American adults in the United States exhibit a robust selection penalty against Asian Americans at the resume screening stage and quantify how cultural-assimilation signals can erase it. In a forced-choice conjoint experiment fielded among about 3,000 U.S. respondents drawn from a Prolific sample and a registered-voter panel, candidates' first names were randomly drawn from a validated dictionary distinguishing Anglicized and Asian-coded individuals. Among Non-Hispanic White respondents — the modal U.S. evaluator and conventional reference group in discrimination research — Asian candidates with Asian-coded first names are selected less often than otherwise-identical White, Black, or Hispanic candidates. The penalty is concentrated in candidates with putatively East Asian and Southeast Asian names; those with South Asian names do not face a penalty, which mirrors observations that South Asians are not underrepresented in leadership positions in the US. Additionally, the penalty appears in all regions of the U.S., and reveals itself among many respondent subgroups — with especially large estimates among Republicans and lower income individuals. This effect is substantively important; consistent with an additive credential model, the penalty against Asians is equivalent to roughly one additional degree step (e.g., BA → MA) or about 1.3 additional years of work experience — placing a hidden tax that Asians must overcome. Pairing our conjoint with unique independent name-perception ratings shows that this discrimination is driven, in part, by perceptions of Asians as noncitizens or perpetual foreigners. Going one step further, we find that cultural-assimilation signals eliminate this gap. Anglicization of the first name closes it entirely: Asian candidates with Anglicized first names are selected by workers for upper management positions at parity with White, Black, and Hispanic candidates. Our results suggest that the modern bamboo ceiling reflects an actionable nameand credential-based cultural-integration filter rather than a generalized racial penalty.
- [6] Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and Maya Lu. "The China Penalty: Measuring and Understanding a Blanket Bias Against Chinese Immigrants Across 22 Nations"
How are Chinese immigrants received in societies that increasingly view their homeland as a threat? Using a large-scale (N ≈ 47,000) conjoint experiment, that presents respondents with profiles of potential immigrants, across 22 of China's geopolitical rivals — spanning Europe, North America, Asia, Latin America, and Africa — we document a consistent and significant penalty against potential Chinese immigrants relative to potential immigrants from non-stigmatized countries. We systematically test each of the leading candidate mechanisms, which include ethnonationalism, chauvinism, and hostility toward foreign investment. None accounts for the penalty. The penalty is diffuse and pervasive, cutting across attitudinal subgroups regardless of respondents' nationalist, chauvinistic, or economic orientations. We further find that conventionally rewarded immigrant attributes such as high skills and host-country language fluency do not differentially benefit Chinese applicants: these premiums hold, but they raise the floor without closing the gap. The Chinese penalty is additive rather than conditional. Individual credentials cannot neutralize a country-of-origin stigma that prevailing theories of immigration attitudes only partially explain.
- [5] Charles Crabtree, John B. Holbein, Bart Bonikowski, and Kiyoteru Tsutsui. "Attention Checks Measure More Than Attention: Comparing Class Bias Across Survey Quality Indicators"
Attention checks are widely used across the social sciences to identify inattentive survey respondents, with researchers routinely excluding those who fail. Using data from 45,401 respondents across 22 countries, we demonstrate that attention check failure strongly correlates with socioeconomic status: excluding failers overrepresents collegeeducated respondents by 9.9 percentage points and higher-income respondents by 0.38 quintiles. Critically, we compare attention checks to an alternative quality indicator—satisficing detection via survey completion time—and find no class bias with the latter approach. This divergence suggests that attention checks conflate data quality with respondent resources, measuring comprehension and literacy as much as attentiveness. We recommend researchers report results stratified by attention check performance rather than excluding failers.
- [4] Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and Surili Sheth. "Forever Foreign? Differential Linguistic Assimilation Standards for Asian Immigrants"
Do Americans hold immigrant groups to different assimilation standards? Using conjoint experiments with 2,999 Americans who evaluated which of 59,980 immigrant profiles was “most assimilated,” we find that linguistic assimilation matters significantly more for Asian immigrants than for other groups. Asian immigrants face a 20% larger penalty (Wave 1) to 65% larger penalty (Wave 2) for speaking their native language at home compared to White immigrants. However, cultural identification with America matters equally across all racial groups. Importantly, this differential is conditional: among Asian immigrants with fluent English, speaking a native language at home is not penalized and may even be valued, whereas those with limited English face severe penalties. These findings suggest that the “perpetual foreigner” stereotype operates through observable behavioral cues interpreted in light of English proficiency, rather than categorically or through internal identities. The results have implications for understanding immigrant incorporation, the boundaries of national identity, and the persistence of racial stereotypes.
- [3] Gaddis, S. Michael, Edvard Nergård Larsen, Charles Crabtree, and John B. Holbein. "How Prevalent is Racial Discrimination Against Black and Hispanic Americans Across Contexts? A Meta-Analysis of Correspondence Audits."
To what extent does racial/ethnic discrimination in America differ across contexts? In this paper, we provide the largest and most comprehensive review of experimental research on racial/ethnic discrimination to date. We conducted a meta-analysis of 78 correspondence audits in the United States, representing over half a million applications, emails, and other forms of correspondence that occur in modern society, including the hiring, housing, medical, public services, and education sectors. Discrimination against racial/ethnic minorities exhibits substantial contextual heterogeneity not recognized in previous discrimination research. We find that racial/ethnic discrimination in the United States continues to be a large problem in some contexts but is less prevalent in other contexts. While discrimination against Black Americans is prevalent in hiring and rental housing, discrimination against Hispanic Americans is only prevalent in hiring. Although discrimination occurs in education, medical, and public services contexts, it is far less common in these sectors. Our findings suggest that discrimination is largest in contexts that are more resource-intensive and have higher short-term economic stakes, despite stronger legal protections against discrimination in those same contexts. Our work confirms that racial/ethnic discrimination in the United States continues to be a pervasive phenomenon that impacts many core parts of the lives of Black and Hispanic Americans and simultaneously reinforces and exacerbates existing inequalities. Our meta-analysis provides a roadmap for the key areas where additional experiments on discrimination are needed.
- [2] John B. Holbein, W. Ben McCartney, and Calvin Zhang. "Development, Exit, and Political Change: Evidence from Opportunity Zones"
Housing shortages have made new construction and residential investment central urban-policy goals, with neighborhood change an important political side effect. We study these consequences using Opportunity Zones, a federal tax incentive for investment in selected low-income tracts, linking election returns to voter files, deeds, and address histories. OZ designation is followed by faster neighborhood growth and rightward changes in local political composition. Decomposition exercises point to exit and replacement rather than incumbent persuasion or turnout: Democrats and renters are less likely to remain in the voter rolls, while later electorates contain disproportionately Republican entrants. The results show that residential mobility is an important channel linking development policy to local political change.
- [1] John B. Holbein, Charles Crabtree, and Zander Furnas. "How Liberal Are Professors? What Existing Measures Can and Cannot Tell Us"
Every available measure of faculty political ideology is methodologically flawed. Surveys, social-media traces, voter files, and campaign contributions all observe selected behaviors, not ideology directly. Each channel conditions on a different form of participation. Response. Platform adoption. Eligibility. Disclosure. Donation. Record linkage. And each form of participation opens the door to external-validity problems, colliders, post-treatment selection, and measurement error. To be clear: the available evidence does point toward a faculty population that leans left of center, especially in the most politically visible groups. What it does not pin down is how far that pattern extends to the much larger share we do not observe, how precise any all-faculty estimate can be, or what would count as enough viewpoint diversity in the first place. We use DAGs, sensitivity analysis, and a selection-aware Bayesian meta-analysis to mark out what these measures actually support. The 2024 donor data observes 13.2 percent of faculty. Conclusions about the other 86.8 percent therefore rest on assumptions, not data. Headline estimates of the faculty left or Democratic share run from 38 percent in voter-registration data to 95.9 percent in 2026 donor data. A 58-point spread. Anchoring on the published literature narrows that gap, but only to 41 points. The Bayesian model puts the underlying issue in the open: under plausible directional selection-bias priors, the donor-channel posterior on all-faculty ideology moves by roughly 31 percentage points, nearly eight times the shift seen in higher-coverage voter-registration data. The bias assumption, not the data, does the inferential work. What we do not observe swamps what we do. We set out what the existing measures support and what they do not. We close with a reporting checklist for estimands, selection mechanisms, normative benchmarks, multi-measure ranges, and sensitivity analyses.
Voting/Elections27 papers
- Holbein, John B. and D. Sunshine Hillygus. Making Young Voters: Converting Civic Attitudes into Civic Action. New York, NY: Cambridge University Press, 2020
In 2016, 90 percent of young Americans showed an interest in politics and 80 percent intended to vote. Yet, only 43 percent of people between the ages of 18 and 29 ended up actually casting a ballot. Making Young Voters investigates what lies at the core of this gap. The authors’ in-depth, interdisciplinary approach reveals that political apathy is not the reason for low levels of youth turnout. Rather, young people too often fail to follow through on their political interests and intentions. Those with “noncognitive” skills related to selfregulation are more likely to overcome internal and external barriers to participation. This book combines theory from psychology, economics, child development, and more to explore possible solutions rooted in civic education and electoral reform. This potentially paradigm-shifting contribution to the literature of American politics serves to influence not only our understanding of voter turnout, but also the fundamental connections between the education system, electoral institutions, and individual civic behavior in a democracy. How young people vote affects not only each individual future, but that of the United States, and of us all.
- Hans Hassell and John B. Holbein. "Changing the Scope of Conflict? Voter Registration and George Floyd’s Murder" Forthcoming, British Journal of Political Science
Prior work has often studied how focusing events – in particular police killings of unarmed people – affect citizens’ attitudes. Do focusing events also affect citizens’ behaviors – changing the scope of conflict by incorporating prospective voters into the political system? We take as an empirical case the murder of George Floyd. Using nationwide voter file data from the United States, we leverage regression discontinuity in time methods to show that Floyd’s murder noticeably increased voter registration. These effects are present across many individual subgroups, providing evidence of mobilization and counter-mobilization. Floyd’s untimely death sparked many people from previously marginalized backgrounds to register, but simultaneously also increased registrations among the politically advantaged – thus mitigating any impact on the partisan balance of power. In short, high-profile traumatic events like Floyd’s death do not appear to disproportionately advantage groups seeking reforms to change the circumstances surrounding such events.
- D'Wayne Bell, Jing Feng, John B. Holbein and Jonathan Smith. "Do STEM Students Vote?" American Educational Research Journal Vol. 61; no. 1 (2024): 48-73
Pundits, politicians, and academics have long worried about potentially low rates of civic participation among STEM-oriented students. Does studying STEM actually decrease the odds that young people will be actively involved in democracy? To answer this question, we created a dataset of over 23 million students in the United States, matched to national validated voting records. This novel dataset is the largest known individual-level dataset in the United States, connecting high school and college students to voting outcomes. It also contains a rich set of demographic and academic variables, to account for many of the common issues related to students' selection into STEM coursework. We consider two measures of STEM participation: Advanced Placement (AP) exam taking in high school and college major. Using both measures, we find that, across model specifications, the estimated relationships between STEM and voting are small in magnitude—about the same effect size as a single get-out-the-vote mailer. Our analyses demonstrate that, on average, marginally more STEM coursework in high school and college does not contribute noticeably to the low voting rates among young people in the United States.
- Tyler Reny, Ben Newman, John B. Holbein, and Hans Hassell. "Public mass shootings cause large surges in Americans' engagement with gun policy." PNAS nexus 2, no. 12 (2023): pgad407.
As public mass shootings continue to plague the United States, a growing scholarly literature seeks to understand the political effects of these tragic events. This literature, however, focuses on public opinion or turnout and vote choice, leaving open to question whether or not public mass shootings affect a range of other important actions citizens may take to engage with gun policy. Leveraging the as-goodas random timing of high-publicity public mass shootings over the past decade and an immense array of publicly available and proprietary data, we demonstrate that these events consistently cause surges in public engagement with gun policy—including internet searches, streaming documentaries, discussion on social media, signing petitions, and donating to political action committees. Importantly, we document the behaviors where shootings induce polarizing upswings in engagement and those where upswings skew toward gun control. Finally, we demonstrate that low-publicity shootings largely exert little-to-no effect on our outcomes.
- Hillygus, D. Sunshine and John B. Holbein. "Refocusing Civic Education to Help Young People Develop the Skills They Need to Engage in Democracy" Annals of the American Academy of Political and Social Science Vol. 705; no. 1 (2023): 73-94
Schools have traditionally taken a "just-the-facts-ma'am" approach to civic education, focusing on governmental structures and political systems. We argue that preparing young people to engage with democracy requires far more than rote memorization of facts and figures. Schools should be laboratories of democracy, where young people's civic intentions are converted into civic behaviors. We argue that to realize that transformation, educators must impart real-world knowledge, practical skills, and nurturing abilities that are not captured by standardized tests of academic achievement: namely, the interpersonal and intrapersonal abilities conducive to civic mindedness. We discuss what these oft-labeled "noncognitive" skills are and how they are measured, review the evidence that shows how they foster democratic participation, articulate a vision for how civics can help develop students' noncognitive skills, and lay out a research agenda for scholars seeking to teach young people the skills requisite to actively participate in democracy.
- Holbein, John B., Marcos Rangel, Raeal Moore, and Michelle Croft. “Is Voting Transformative? Expanding and Meta-Analyzing the Evidence” Political Behavior Vol. 45, no. 3 (2023): 1015-1044
Voting is the foundational act of democracy. While thousands of studies have treated voting as a dependent variable, comparatively little research has studied voting as an independent variable. Here we flip the causal arrow and explore the effect of exogenous voting shocks on citizens’ broader attitudes and behaviors. To do so, we first use two waves from a uniquely large survey of young people in the United States, pairing this with a regression discontinuity design. We augment these results with a new meta-analysis of all causally-identified studies exploring whether voting is transformative. We find that—despite voting at much higher rates—individuals induced to vote, regardless of the mode used to mobilize, are (precisely) no different from all-else-equal individuals that are not. Our results illuminate the (non)consequences of a vitally important—and widely studied—political behavior and speak to the broader importance of voting as an object of study.
- Mo, Cecilia Hyunjung, John B. Holbein, and Elizabeth Mitchell Elder. "Civilian National Service Programs Can Powerfully Increase Youth Voter Turnout." Proceedings of the National Academy of Sciences. Vol 119, no. 29 (2022): e2122996119
Low rates of youth voting are a feature of contemporary democracies the world over, with the United States having some of the lowest youth turnout rates in the world. However, far too little is known about how to address the dismal rates of youth voter participation found in many advanced democracies. In this paper, we examine the causal effect of a potentially scalable solution that has attracted renewed interest today: voluntary national service programs targeted at the youth civilian population. Leveraging the large pool of young people who apply each year to participate in the Teach For America (TFA) program—a prominent voluntary national service organization in the United States that integrates college graduates into teaching roles in low-income communities for 2 y—we examine the effect of service participation on voter turnout. To do so, we match TFA administrative records to large-scale nationwide voter files and employ a fuzzy regression discontinuity design around the recommended admittance cutoff for the TFA program. We find that serving as a teacher in the Teach For America national service program has a large effect on civic participation—substantially increasing voter turnout rates among applicants admitted to the program. This effect is noticeably larger than that of previous efforts to increase youth turnout. Our results suggest that civilian national service programs targeted at young people have great promise in helping to narrow the stubborn and enduring political engagement gap between younger and older citizens.
- Barber, Michael, and John B. Holbein. "400 million voting records show profound racial and geographic disparities in voter turnout in the United States." PloS one 17, no. 6 (2022): e0268134.
One of the core tenets of a well-functioning representative democracy is that the people who vote to elect government officials are representative of the public. Here we reinforce the idea that reality is far from this lofty ideal. We document the extent and nature of inequities in voter participation in the United States with a level of granularity and precision that previous research has not afforded. To do so, we use a unique nationwide dataset of approximately 400 million validated voting records across multiple election cycles. With this novel dataset, we document large and persistent gaps in voter turnout by race, age, and political affiliation. Minority citizens, young people, and those who support the Democratic Party are much less likely to vote than whites, older citizens, and Republican Party supporters. Minorities, youth, and democrats are also much more likely to live in local communities where fewer individuals vote—areas that we term turnout deserts. Turnout deserts are especially pernicious given that they are self-reinforcing—bolstered by the social dynamics that fundamentally shape citizens' voting patterns. Our results show just how glaring inequities in political participation are in the US. These patterns threaten the very fabric of our democracy and fundamentally shift the balance of political power in the halls of government towards the interests of whites, older citizens, and republicans. They illustrate that participation in the United States is strikingly unequal—far from the ideals that this country has long aspired to.
- Holbein, John B., Catherine Bradshaw, Nicholas Ialongo, B. Kal Munis, Jill Rabinowitz. "Promoting Voter Turnout: An Unanticipated Impact of Early Childhood Preventive Interventions." Prevention Science. Vol. 23 (2022): 192-203
Preventive interventions in early childhood have a range of behavioral and health effects. However, there is an emerging literature extending this work to include acts of civic engagement, such as voting. Given that America has one of the lowest and most disparate rates of voter turnout in the world—and most of the current efforts aimed at boosting voter turnout and making the electorate more representative of the general public are proximal to the voting experience—there is a need for a better understanding of the potential long-term impact of early-childhood programming on civic engagement in adulthood. This paper builds on theories of political socialization and prior research demonstrating significant impacts of the Fast Track preventive intervention on voter turnout to examine the extent to which there are positive impacts on voter participation for other evidence-based preventive interventions targeting children's social and emotional capacities. Specifically, we leveraged data from a randomized controlled trial of the classroom-centered (CC) and the family school partnership (FSP) preventive interventions delivered in first grade. We analyzed data from approximately 700 urban, predominately African American, public school students who were randomly assigned to classrooms that either implemented (1) the classroom-centered intervention (which included the good behavior game), (2) the FSP intervention, or (3) the business as usual (i.e., control group). Data from the trial were combined with archival voter data when the youth were in their early 30s. Analyses demonstrated positive impacts of the CC preventive intervention on voter turnout more than two decades after exposure to the prevention program. Taken together, these findings provide additional evidence that some of the attributes that promote active participation in democracy can be fostered in early childhood—long before most interventions that have previously tried, and often failed, to increase voter turnout.
- Easton, Matt and John B. Holbein. "The Democracy of Dating: How Political Affiliations Shape Relationship Formation." Journal of Experimental Political Science. Vol. 8, no. 3 (2021): 260-272.
How much does politics affect relationship building? Previous experimental studies have come to vastly different conclusions – ranging from null to truly transformative effects. To explore these differences, this study replicates and extends previous research by conducting five survey experiments meant to expand our understanding of how politics does/does not shape the formation of romantic relationships. We find that people, indeed, are influenced by the politics of prospective partners; respondents evaluate those in the political outgroup as being less attractive, less dateable, and less worthy of matchmaking efforts. However, these effects are modest in size – falling almost exactly in between previous study estimates. Our results shine light on a literature that has, up until this point, produced a chasm in study results – a vital task given concerns over growing levels of partisan animus in the USA and the rapidly expanding body of research on affective polarization.
- Barber, Michael and John B. Holbein. "The Participatory and Partisan Impacts of Mandatory Vote-by-Mail" Science Advances Vol. 6, no. 35 (2020): eabc7685
Recently, mandatory vote-by-mail has received a great deal of attention as a means of administering elections in the United States. However, policy-makers disagree on the merits of this approach. Many of these debates hinge on whether mandatory vote-by-mail advantages one political party over the other. Using a unique pairing of historical county-level data that covers the past three decades and more than 40 million voting records from the two states that have conducted a staggered rollout of mandatory vote-by-mail (Washington and Utah), we use several methods for causal inference to show that mandatory vote-by-mail slightly increases voter turnout but has no effect on election outcomes at various levels of government. Our results find meaning given contemporary debates about the merits of mandatory vote-by-mail. Mandatory vote-by-mail ensures that citizens are given a safe means of casting their ballot while simultaneously not advantaging one political party over the other.
- Hassell, Hans, John B. Holbein, and Matthew Baldwin. "Mobilize for Our Lives? School Shootings and Democratic Accountability in U.S. Elections" American Political Science Review. Vol. 114, no. 4 (2020): 1375-1385
Gun violence is a large and growing problem in the United States. Many reformers look towards elections to spur policy change in this area. In this paper, we explore the effects of school shootings on electoral mobilization and election outcomes. We pair data from several sources that measure validated voter registration; validated voter turnout; and the electoral performance of officials at the local, state, and federal levels with regression discontinuity and panel methods. Our effects show that shootings have little to no effect on electoral outcomes in the United States. Our work demonstrates that even when tragic events occur that are squarely in the realm of elected officials' responsibility, have high levels of issue salience, are highly-covered by the media, draw citizens' attention, and (perhaps) shift public opinion, these seemingly favorable conditions may not be enough to elicit democratic accountability.
- Holbein, John B. and Marcos Rangel. "Does Voting Have Upstream and Downstream Consequences? Regression Discontinuity Tests of the Transformative Voting Hypothesis." The Journal of Politics. Vol. 82, no. 4 (2020): 1196-1216
Voting is a central pillar of political science research; indeed, scholars have long addressed questions like, “Who votes?,” “Why do people vote?,” and “What interventions increase voting?” However, only a few have considered whether voting changes adjacent civic dispositions and behaviors. In this paper, we explore the effects of voting shocks on young citizens’ political interest, memberships, social awareness, and political knowledge in the lead up to the voting experience (“upstream”) and in the months and years after (“downstream”). To do so, we use a unique combination of large survey data from two countries paired with an exact date-of-birth regression discontinuity design. We find that eligibility to vote voluntarily and exposure to compulsory voting—despite eliciting large turnout increases—have precisely estimated null effects on young people’s upstream or downstream civic engagement. While voting may be an important experience, it appears to have smaller transformative effects than previously thought.
- Holbein, John B., D. Sunshine Hillygus, Matthew A. Lenard, Christina Gibson-Davis, Darryl V. Hill, and. "The Development of Students’ Engagement in School, Community, and Democracy" British Journal of Political Science. Vol. 50, no. 4 (2020): 1439-1457
This article explores the origins of youth engagement in school, community and democracy. Specifically, it considers the role of psychosocial or non-cognitive abilities, like grit or perseverance. Using a novel original large-scale longitudinal survey of students linked to school administrative records and a variety of modeling techniques – including sibling, twin and individual fixed effects – the study finds that psychosocial abilities are a strong predictor of youth civic engagement. Gritty students miss less class time and are more engaged in their schools, are more politically efficacious, are more likely to intend to vote when they become eligible, and volunteer more. Our work highlights the value of psychosocial attributes in the political socialization of young people.
- Hassell, Hans, John B. Holbein, and Matt Miles. "There Is No Liberal Media Bias in Which News Stories Political Journalists Choose to Cover" Science Advances Vol. 6, no. 14 (2020): eaay9344
Is the media biased against conservatives? Although a dominant majority of journalists identify as liberals/Democrats and many Americans and public officials frequently decry supposedly high and increasing levels of media bias, little compelling evidence exists as to (i) the ideological or partisan leanings of the many journalists who fail to answer surveys and/or identify as independents and (ii) whether journalists' political leanings bleed into the choice of which stories to cover that Americans ultimately consume. Using a unique combination of a large-scale survey of political journalists, data from journalists' Twitter networks, election returns, a large-scale correspondence experiment, and a conjoint survey experiment, we show definitively that the media exhibits no bias against conservatives (or liberals for that matter) in what news that they choose to cover. This shows that journalists' individual ideological leanings have unexpectedly little effect on the vitally important, but, up to this point, unexplored, early stage of political news generation.
- Akee, Randall, William Copeland, John B. Holbein, and Emilia Simeonova. "Human Capital and Voting Behavior across Generations: Evidence from an Income Intervention." American Political Science Review 114, no. 2 (2020): 609-616
Despite clear evidence of a sharp income gradient in voting participation, it remains unclear whether income truly causes voting. In this article, we investigate how exogenous increases in unearned income affect voting in U.S. elections for two generations (parents and children) from the same household. In contrast to predictions made by current models of voting, we find the income shock had no effect on parents' voting behaviors. However, we also find that increasing household income has heterogeneous effects on the civic participation of children from different socioeconomic backgrounds. It increases children's voting propensity among those raised in initially poorer families—resulting in substantially narrowed participatory gaps. Our results are consistent with a more nuanced view of how individual resources affect patterns of voting than the dominant theoretical framework of voting—the resource model—allows. Voting is fundamentally shaped by the human capital accrued long before citizens are eligible to vote.
- Dynes, Adam and John B. Holbein. "Noisy Retrospection: The Effect of Party Control on Policy Outcomes" American Political Science Review 114, no. 1 (2020): 237-257.
Retrospective voting is vital for democracy. But, are the objective performance metrics widely thought to be relevant for retrospection—such as the performance of the economy, criminal justice system, and schools, to name a few—valid criteria for evaluating government performance? That is, do political coalitions actually have the power to influence the performance metrics used for retrospection on the timeline introduced by elections? Using difference-in-difference and regression discontinuity techniques, we find that US states governed by Democrats and those by Republicans perform equally well on economic, education, crime, family, social, environmental, and health outcomes on the timeline introduced by elections (2–4 years downstream). Our results suggest that voters may struggle to truly hold government coalitions accountable, as objective performance metrics appear to be largely out of the immediate control of political coalitions.
- Schafer, Jerome and John B. Holbein. "When Time Is of the Essence: A Natural Experiment on How Time Constraints Influence Elections" The Journal of Politics 82, no. 2 (2020): 418-432
Foundational theories of voter turnout suggest that time is a key input in the voting decision, but we possess little causal evidence about how this resource affects electoral behavior. In this article, we use over two decades of elections data and a novel geographic regression discontinuity design that leverages US time zone boundaries. Our results show that exogenous shifts in time allocations have significant political consequences. Namely, we find that citizens are less likely to vote if they live on the eastern side of a time zone border. Time zones also exacerbate participatory inequality and push election results toward Republicans. Exploring potential mechanisms, we find suggestive evidence that these effects are the consequence of insufficient sleep and moderated by the convenience of voting. Regardless of the exact mechanisms, our results indicate that local differences in daily schedules affect how difficult it is to vote and shape the composition of the electorate.
- Fraga, Bernard and John B. Holbein. "Measuring Youth and College Student Voter Turnout." Electoral Studies 65, no. 3 (2020).
Young adulthood is a critical period in civic development. However, measuring electoral participation among this group generally—and the many young people who go to college in particular—is fraught with potential pitfalls stemming from a reliance on survey-based measures of voting. In this note, we compare patterns of youth turnout in two large-scale, survey-based datasets commonly used to measure voting, the Current Population Survey and the Cooperative Congressional Election Study, to two voter-file based datasets: the National Study of Learning, Voting, and Engagement (NSLVE) and a comprehensive nationwide voterfile provided by the Data Trust. We find high levels of concordance between measures in the NSLVE, Data Trust, and the CPS. However, despite linking their sample to validated voter records, the CCES does not mirror these benchmarks. We conclude by discussing the challenges and opportunities that shape the study of youth turnout.
- Holbein, John B. “Childhood Skill Development and Adult Political Participation.” American Political Science Review 111, no. 3 (2017): 572-583.
Recent child development research shows that the psychosocial or noncognitive skills that children develop—including the ability to self-regulate and integrate in social settings—are important for success in school and beyond. Are these skills learned in childhood also important for adult political behaviors like voting? In this article, I use a unique school-based 20-year field experiment to explore whether children who develop psychosocial skills early on are more likely to vote in adulthood than those who do not. Matching subjects to voter files, I show that this intervention had a noticeable long-run impact on political participation. These results highlight the need to better understand how childhood experiences shape civic behaviors later in life. During this critical period, children can be taught the not explicitly political, but still vital, skills that set them on a path toward political participation in adulthood.
- Holbein, John B. “Left Behind? Citizen Responsiveness to Government Performance Information.” American Political Science Review 110, no. 2 (2016): 353-368.
Do citizens respond to policy-based information signals about government performance? Using multiple big datasets—which link for the first time large-scale school administrative records and individual validated voting behavior—I show that citizens react to exogenous school failure signals provided by No Child Left Behind. These signals cause a noticeable increase in turnout in local school board elections and increase the competitiveness of these races. Additionally, I present evidence that school failure signals cause citizens to vote with their feet by exiting failing schools, suggesting that exit plays an underexplored role in democratic accountability. However, performance signals elicit a response unequally, with failure primarily mobilizing high propensity citizens and encouraging exit among those who are white, affluent, and more likely to vote. Hence, while performance signals spur a response, they do so only for a select few, leaving many others behind.
- Holbein, John B. and D. Sunshine Hillygus. "Making Young Voters: The Impact of Preregistration on Youth Turnout." American Journal of Political Science 60, no. 2 (2016): 364-382 [Correction] [Additional analysis].
Recent research has cast doubt on the potential for various electoral reforms to increase voter turnout. In this article, we examine the effectiveness of preregistration laws, which allow young citizens to register before being eligible to vote. We use two empirical approaches to evaluate the impact of preregistration on youth turnout. First, we implement difference-in-difference and lag models to bracket the causal effect of preregistration implementation using the 2000–2012 Current Population Survey. Second, focusing on the state of Florida, we leverage a discontinuity based on date of birth to estimate the effect of increased preregistration exposure on the turnout of young registrants. In both approaches, we find preregistration increases voter turnout, with equal effectiveness for various subgroups in the electorate. More broadly, observed patterns suggest that campaign context and supporting institutions may help to determine when and if electoral reforms are effective.
- D'Wayne Bell, John B. Holbein, Sam Imlay, and Jonathan Smith. "Which Colleges Increase Voting Rates?" Revised and Resubmitted at the American Economic Journal: Applied Economics
Using millions of SAT takers linked to college enrollment and voting records, we document wide variation in voter participation across otherwise similar colleges. We estimate over 2,600 colleges' value-added to the likelihood of voting and find that 30 percent of the observed variation across colleges is attributable to the colleges themselves. The standard deviation of these estimates ranges from 3 to 5 percentage points (6–10 percent), with almost all students in our sample attending colleges that increase their entrants' probability of voting. These effects appear soon after enrollment, persist beyond college, and are driven by colleges creating new voters.
- Michael Bloem, John B. Holbein, Sam Imlay, and Jonathan Smith. “Voting Among Siblings” Invitation to Revise and Resubmit at American Economic Journal: Economic Policy
Using millions of siblings in the U.S., we detail three findings that quantify whether siblings influence one another to vote in national elections. First, and descriptively, younger siblings are 10 percentage points (50 percent) more likely to vote in their first eligible election when their older sibling votes in a prior election. Second, roughly one-third of this is caused by the older sibling voting, as determined by age-of-voting-eligibility thresholds in a regression discontinuity design. Third, the causal impact of a sibling voting runs in the other direction as well—younger siblings increase the probability of their older siblings voting in their early 20’s by 14 percent. These results demonstrate the influence and importance of family and peers in creating an engaged citizenry and underscore that across a wide array of policy domains, conventional impact evaluations do not fully account for all of policies’ impacts.
- Hans Hassell, Taylor Hill, and John B. Holbein. "Do ICE Arrests Spark Electoral Action?" Under Review
Does U.S. Immigration and Customs Enforcement (ICE) activity trigger electoral mobilization? Theory and prior evidence suggest it should, as deportation imposes exceptionally high stakes on affected communities. Indeed, contemporary enforcement has sparked widespread protests. But do these actions translate into electoral engagement? The second Trump administration’s enforcement surge represents the most favorable conditions ever observed for detecting an electorally focused response. Using a nationwide panel from 2023 to 2025, we test whether ICE activity has translated into changes in voter registration and political donations via difference-in-differences, event study, and regression discontinuity designs. We find largely null or substantively small estimates. We can rule out even very modest changes in voter registration patterns overall and across subgroups, while donation effects are substantively negligible and offsetting across parties. Results survive a demanding critical-case test using Minnesota ICE activity in late 2025. Even historically unprecedented enforcement does not automatically translate into political mobilization.
- Hans Hassell, John B. Holbein, Kevin Reuning, and Cindy West. "The Electoral Consequences of Social Protest in Contemporary America" Under Review
Do protest-led social movements change elections? Virtually all prior research says yes, and markedly so. However, this research has often struggled to identify causal effects or, even when it has, tended to examine singular, high-salience movements, leaving open what happens when protest is a sustained, system-wide phenomenon. We address this gap by focusing on the United States and merging protest event data from the Crowd Counting Consortium, which captures nearly 200,000 events from small local gatherings to mass mobilizations across every ideological stripe, with nationwide voter files, turnout and election returns, and FEC donation records from 2017 through 2024. Using difference-in-differences and event study techniques, we find effects that are, with few exceptions, small at best and broadly indicative of a null effect. This null holds across protests that span the ideological spectrum, the target and size of the protest, and is consistent across demographic subgroups. The limited exceptions are revealing: in the rare instances where confrontational protests occur—resulting in injuries or property destruction and comprising no more than ≈ 1% of all protest events—we observe a modest bipartisan donation surge to both Democratic and Republican candidates, with no consistent net advantage for either party. In subsequent elections, we also observe a small and somewhat sensitive to model specification increase in Democratic vote share. Overall, however, in a deeply polarized era, we find that protest effects on elections are muted.
- John B. Holbein, W. Ben McCartney, and Calvin Zhang. "Development, Exit, and Political Change: Evidence from Opportunity Zones"
Housing shortages have made new construction and residential investment central urban-policy goals, with neighborhood change an important political side effect. We study these consequences using Opportunity Zones, a federal tax incentive for investment in selected low-income tracts, linking election returns to voter files, deeds, and address histories. OZ designation is followed by faster neighborhood growth and rightward changes in local political composition. Decomposition exercises point to exit and replacement rather than incumbent persuasion or turnout: Democrats and renters are less likely to remain in the voter rolls, while later electorates contain disproportionately Republican entrants. The results show that residential mobility is an important channel linking development policy to local political change.
Discrimination18 papers
- Crabtree, Charles and John B. Holbein. Explaining Discrimination. Under advanced contract with Cambridge University Press
- Charles Crabtree, John B. Holbein, Mitchell Bosley, and Semra Sevi. “Can AI Help Reduce Prejudice? Evaluating the Effectiveness of AI-Powered Personalized Persuasion on Support for Transgender Rights” Forthcoming, PNAS Nexus
Personalized interpersonal conversations are among the most effective known tools for reducing prejudice, yet they are difficult to scale because they require skilled human facilitators. This study tests whether artificial intelligence can approximate the effects of these interventions. Using OpenAI's GPT-4o, we developed a messaging-based intervention that engaged U.S. participants in individualized, morally aligned dialogues about transgender rights. In a preregistered experiment, these AI-mediated conversations significantly increased support for transgender rights across multiple attitudinal measures. Robustness checks, including weighting and sensitivity analyses, confirmed the reliability of these effects, and analyses of the conversations support the idea that moral matching between the AI and participants played a key role in reducing prejudice. However, follow-up data collected one week later indicated that the attitudinal gains diminished over time, suggesting that reinforcement may be necessary to sustain change. Together, these findings indicate that generative AI can facilitate value-aligned dialogue capable of shifting social attitudes, while highlighting the challenge of achieving durable impacts.
- Gaddis, S. Michael, Natasha Quadlin, Charles Crabtree, John B. Holbein, and Edvard Nergård Larsen. "Missing Results of Discrimination: A Systematically Comparative Meta-Analysis and Re-Examinations of Racial, Gender, and Intersectional Discrimination using Correspondence Audits" Conditionally Accepted, Sociological Science
Intersectionality theory argues that race and gender cannot be fully understood in isolation or through an additive approach. However, field experiments frequently examine aggregate racial or gender discrimination without accounting for within-category differences (e.g., gendered variations in racial discrimination) or intersectionality. Building on theoretical perspectives of intersectionality, we introduce a systematically comparative analytic framework designed to (a) identify missing results of correspondence audits and (b) provide recommendations to help scholars conduct more holistic analyses. We conduct a meta-analysis of 52 correspondence audits, encompassing nearly 330,000 tests for discrimination, and a re-examination of each study using seven discrimination ratios. The meta-analysis reveals a previously overlooked pattern in rental housing discrimination: compared to White men, Black men experience discrimination, Black women's outcomes are statistically similar, and White women receive preferential treatment. Additionally, our re-examination uncovers ten ways scholars may unintentionally miss key findings when failing to adopt a systematically comparative intersectional approach. We conclude with best-practice recommendations to guide scholars in designing, analyzing, and citing correspondence audits, helping them avoid these problematic scenarios. Importantly, our framework extends beyond field experiments focused on race and gender and can be broadly applied to research on disparities, enabling more comprehensive analyses across numerous dimensions.
- Gaddis, S. Michael, Charles Crabtree, John B. Holbein, and Edvard Nergård Larsen. "Racially Distinctive Names Signal Race and Class" Sociological Science. Vol. 9 (2022): 454-472
Researchers studying discrimination and bias frequently conduct experiments that use racially distinctive names to signal race or ethnicity. The evidence that these studies provide about racial discrimination depends on the assumption that the names researchers use differ only based on perceived race and not some other factor. In this article, we assess this common assumption using data from five different studies (n = 1,004; 2,002; 1,035; 5,631; 1,858) conducted at different times across four separate survey platforms (Lucid Marketplace, Lucid Theorem, MTurk, and Prolific). We find evidence that names commonly used to signal race/ethnicity also influence perceptions about socioeconomic status and social class. Specifically, we observe that Americans tend to think that individuals with names typically used by Black and Hispanic people have lower educational attainment and income and are of a lower social class. Even when we present respondents with the educational attainment of a named individual, respondents still perceive Black people as lower social class than White people. We discuss the implications of these findings for past and future experimental work that uses names to signal race. We also articulate the importance of choosing names that best approximate the quantity that scholars want to estimate.
- Block, Ray, Charles Crabtree, John B. Holbein, and Quin Monson. "Conceptual and Empirical Issues that Arise When Using Correspondence Audits to Measure Racial Discrimination." Proceedings of the National Academy of Sciences. Vol. 119, no. 38 (2022): e2210695119
Mitterer raises important issues in response to our work, but his conclusions are incorrect and incomplete. How we conceptualize and design experiments to test for discrimination matters a great deal. We agree that experiments using names as racial signals to study discrimination should consider what other characteristics beyond race people infer from names. We also agree that multiple names should be used; our work used five times the number of names commonly used, and these names were preregistered prior to data collection. Future work should continue to use multiple names but should do so with care. We disagree, however, with Mitterer's analytic approach. He analyzes our data as if it were collected with a between-subjects design when we actually used a within-subjects design. Mitterer ignores the considerable statistical-power gains that come from a within-subjects design. Mitterer's model with individual fixed effects—as is standard in within-subjects designs—shows robust evidence for discrimination, with effect sizes that are substantively and statistically indistinguishable from those provided in our original paper. We are ambivalent as to whether Mitterer's approach is best. We acknowledge model-based uncertainty. However, any analysis must correctly account for experimental design.
- Block, Ray, Charles Crabtree, John B. Holbein, and Quin Monson. "Are Americans Less Likely to Reply to Emails from Black People Relative to White People?" Proceedings of the National Academy of Sciences. Vol. 118, no. 52 (2021): e2110347118
In this article, we present the results from a large-scale field experiment designed to measure racial discrimination among the American public. We conducted an audit study on the general public—sending correspondence to 250,000 citizens randomly drawn from public voter registration lists. Our within-subjects experimental design tested the public's responsiveness to electronically delivered requests to volunteer their time to help with completing a simple task—taking a survey. We randomized whether the request came from either an ostensibly Black or an ostensibly White sender. We provide evidence that in electronic interactions, on average, the public is less likely to respond to emails from people they believe to be Black (rather than White). Our results give us a snapshot of a subtle form of racial bias that is systemic in the United States. What we term everyday or "paper cut" discrimination is exhibited by all racial/ethnic subgroups—outside of Black people themselves—and is present in all geographic regions in the United States. We benchmark paper cut discrimination among the public to estimates of discrimination among various groups of social elites. We show that discrimination among the public occurs more frequently than discrimination observed among elected officials and discrimination in higher education and the medical sector but simultaneously, less frequently than discrimination in housing and employment contexts. Our results provide a window into the discrimination that Black people in the United States face in day-to-day interactions with their fellow citizens.
- Pfaff, Steven, Charles Crabtree, Holger L. Kern, and John B. Holbein. "Do Street‐Level Bureaucrats Discriminate Based on Religion? A Large‐Scale Correspondence Experiment among American Public School Principals" Public Administration Review. Vol. 81, no. 2 (2021): 244-259
Although public administration scholars have long studied discrimination on the basis of race/ethnicity, class, and gender, little to no research exists on whether street-level bureaucrats provide differential services based on the religious identity of their constituents. This article reports the results from a large-scale correspondence study of street-level bureaucrats in the American public school system. The authors emailed the principals of a large sample of public schools and asked for a meeting, randomly assigning the religious (non)affiliation of the family. To get at potential causal mechanisms, religious belief intensity was also randomly assigned. The findings show evidence of substantial discrimination against Muslims and atheists on a par with, and sometimes larger than, the racial discrimination found in previous studies. These individuals are substantially less likely to receive a response, with discrimination growing when they signal that their beliefs are more intense. Protestants and Catholics face no discrimination unless they signal that their religious beliefs are intense.
- Carnes, Nicholas and John B. Holbein. "Do Public Officials Exhibit Social Class Biases when they Handle Casework? Evidence from Multiple Correspondence Experiments" PLOS ONE 14, no. 3 (2019).
Are public officials more responsive to requests from affluent or poor constituents? A growing body of evidence suggests that lawmakers are more responsive to the rich when they craft policy. However, some scholars theorize that officials also exhibit a corresponding bias in favor of the poor when they handle casework, essentially giving policy to the rich and services to the poor. In this paper, we test this casework prediction using four experiments in which confederates sent simple requests to state or local officials. In each, our confederates' reported social classes were randomly assigned and signaled with a brief introductory statement mentioning the sender's occupation or economic situation. Across our samples, we find precisely-estimated null effects of social class biases: the officials we studied were equally likely to respond regardless of the constituent's class. These findings raise doubts about whether casework is really a class-biased process.
- Holbein, John B. and Hans J.G. Hassell. "When Your Group Fails: The Effect of Race-Based Performance Signals on Citizen Voice and Exit" Journal of Public Administration Research and Theory 29, no. 2 (2019): 268–286.
In recent years, performance-based accountability reforms have become widespread—particularly in the public education system—as a means of enhancing service delivery and citizens' knowledge of and engagement with government. While an ever-growing literature has considered the overall effects of these administrative systems, few have fully considered the effect these have on underlying subgroups of relevance and inequalities in responses across these groups. In this article, we examine how citizens of various racial subgroups respond when governments send negative race-based signals about the performance of inand out-racial groups. Specifically, we explore whether No Child Left Behind's (NCLB) race-based failure signals affect racial groups' use of voice in school board elections and their use of exit from local public schools. To do so, we combine school and voter administrative records in North Carolina with a regression discontinuity design that leverages exogenous variation around NCLB's race-based school failure cutoffs. Consistent with our theoretical framework, we find that white and black citizens respond differently to race-based failure signals. We find the whites are more responsive overall: responding to failure signals of various types both by voting in local school board elections and exiting local failing schools. African Americans, however, seem much more sensitive to racialized cues—responding at the ballot box but not by exiting when their group fails. These results show that while performance accountability systems have broader effects than previously realized: shifting the composition of local elections and distorting the racial makeup of schools.
- Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and J. Quin Monson. “Micro-Inequities Against Asian Americans: Evidence from a Large-Scale Field Experiment” Revised and Resubmitted at the American Political Science Review
To what extent do Asians face micro-inequities, and can signaling cultural assimilation reduce this bias? We examine these questions through a field experiment involving 250,000 randomly sampled registered voters in the United States. Participants received emails with a simple request for help from senders whose randomized racial identity (Asian or white) was signaled through their names. Among senders with putatively Asian names, we randomized whether the first name was Western or anglicized — a practice often known as "name whitening." We find that micro-inequities against Asians are (1) large, precisely estimated, and robust; (2) systemic across the U.S.; (3) common across most demographic subgroups; and (4) uncorrelated with previously documented overt forms of anti-Asian bias. Leveraging both the field experiment and original survey data on how names are perceived, we test nine potential mechanisms: perceptions of Asian identity, perceived foreignness, perceived socioeconomic status, perceived character flaws, perceptions of name inauthenticity toward Asian-origin names (i.e., the inclination to regard such names as artificial or automated ("bot") identities rather than real individuals), filtering by spam detection algorithms, a lack of potential exposure to Asians, high affectedness to COVID-19, and high economic resentment toward China. Our results are consistent with discrimination against Asians being driven, in large part, by perceptions of foreignness or non-citizenship and, secondarily, the misperception that Asian-origin names are not associated with real people. While anglicizing Asian names reduces the discrimination we detected by about half, it does not eliminate it. These results provide some of the clearest evidence to date on the nature and causes of micro-inequities faced by Asians in the United States.
- Charles Crabtree, S. Michael Gaddis, and John B. Holbein. "How Should Researchers Signal Race in Correspondence Audits? Considering The Validity of Names and Photographs" Rejection with an Invitation to Resubmit at the Journal of Politics
How should researchers signal race in correspondence audits of discrimination? In a forthcoming paper at the Journal of Politics, Kaufman, Celaya, and Grumbach (hereafter KCG) argue that researchers should use photos rather than names to signal race. They claim that name-based treatments suffer from “treatment non-compliance” and propose using photos as a solution. However, we show this claim is neither novel nor their solution effective. Using a national, quota-based survey of 1,400 U.S. adults, we demonstrate that respondents frequently perceive emails with photos as spam, that senders rarely include photos in many real-world communication settings, and that the images used by KCG send bundled and unpredictable signals beyond race. Together with our analytical critique, this empirical evidence suggests KCG’s approach lowers ecological validity and signal clarity and does not improve signal strength. We recommend caution and propose alternative improvements for experimental discrimination research.
- Michael Gaddis, Charles Crabtree, John B. Holbein, and Steve Pfaff. "Racial/Ethnic Discrimination in Interactions with the U.S. Public Education System: A Correspondence Audit of Principals." Invitation to Revise and Resubmit at Sociology of Education
Although numerous studies document different forms of discrimination in the U.S. public education system, very few provide plausibly causal estimates. Thus, it is unclear to what extent public school principals discriminate against racial and ethnic minorities. Moreover, no studies test for heterogeneity in racial/ethnic discrimination by individual-level resource needs and school-level resource strain – potentially important moderators in the education context. Using a correspondence audit, we examine bias against Black, Hispanic, and Chinese American families in interactions with 52,792 public K-12 principals in 33 states. Our research provides causal evidence that Hispanic and Chinese American families face significant discrimination in initial interactions with principals, regardless of individual-level resource needs. Black families, however, only face discrimination when they have high resource needs. Additionally, principals in schools with greater resource strain discriminate more against Chinese American families. This research uncovers complexities of racial/ethnic discrimination in the K-12 context because we examine multiple racial/ethnic groups and test for heterogeneity across individualand school-level variables. These findings highlight the need for researchers conducting future correspondence audits to expand the scope of their research to provide a more comprehensive analysis of racial/ethnic discrimination in the U.S.
- Michael Gaddis, Charles Crabtree, John B. Holbein, and Steve Pfaff. "Local Political Party Voting Context Moderates Public School Principals’ Levels of Racial/Ethnic Discrimination" Invitation to Revise and Resubmit at the Journal of Race, Ethnicity, and Politics
Correspondence audits document causal evidence of racial/ethnic discrimination in many contexts. However, few studies have examined whether local political party voting context influences individuals to engage in “stakeholder-centric” discrimination on behalf of or in response to expectations of others. We examine heterogeneity in racial/ethnic discrimination by the county-level Republican vote share with a correspondence audit of 52,792 K-12 public-school principals across 33 states. We email principals posing as parents considering a move to the school district and requesting a meeting. We find evidence that the county-level Republican vote share strongly moderates racial/ethnic discrimination against Black and Chinese American families. While all groups are less likely to receive a response from principals as the Republican vote share increases, the declines for Black and Chinese American families are largest. Thus, discrimination against Black and Chinese American families is sizable in counties with the highest Republican vote share. These findings shed light on how partisanship can shape the experiences of historically marginalized groups. Furthermore, there may be benefits to targeting limited resources to geographies where discrimination is more likely to occur.
- Charles Crabtree, Michael Gaddis, and John B. Holbein. "Shifting Paradigms: Nine Decades of Identity and Discrimination Scholarship in Academic Research" Under Review
How has academic attention to identity and discrimination evolved over time? Analyzing 216,500 English-language research articles from JSTOR (1933–2025), we document temporal trends in twelve identity categories: race, ethnicity, gender, sex, class, religion, immigration, disability, sexuality, indigenous peoples, minority status, and discrimination. Three major transformations emerge. First, a "gender turn" beginning in the 1970s saw gender keywords rise from 1% to 11% by 2020, while sex-based framing declined from 40% (1980s peak) to 25% today. Second, attention to class peaked at 29% during the 1970s economic crises but declined through the neoliberal era to 17% by 2020. Third, immigration scholarship surged from 6% to 12% (2000–2020). We also document emerging categories: disability (1% to 3%), sexuality (0% to 3% since 1990), and indigenous peoples (3% to 8%). Secondary analysis of 20,706 race/ethnicity articles reveals that "White" keywords mark Critical Whiteness Studies (90.5% co-occur with "race"), not reverse discrimination claims, while Asian American scholarship reflects post-1965 immigration and model minority debates. Together, these patterns provide a systematic longitudinal account of how identity categories enter, expand within, and recede from the scholarly agenda.
- Charles Crabtree, John B. Holbein, and Cecilia Hyunjung Mo. "Cultural Assimilation Weakens Workers’ Discriminatory Preferences Against Potential Asian Bosses"
We document that a large sample of American adults in the United States exhibit a robust selection penalty against Asian Americans at the resume screening stage and quantify how cultural-assimilation signals can erase it. In a forced-choice conjoint experiment fielded among about 3,000 U.S. respondents drawn from a Prolific sample and a registered-voter panel, candidates' first names were randomly drawn from a validated dictionary distinguishing Anglicized and Asian-coded individuals. Among Non-Hispanic White respondents — the modal U.S. evaluator and conventional reference group in discrimination research — Asian candidates with Asian-coded first names are selected less often than otherwise-identical White, Black, or Hispanic candidates. The penalty is concentrated in candidates with putatively East Asian and Southeast Asian names; those with South Asian names do not face a penalty, which mirrors observations that South Asians are not underrepresented in leadership positions in the US. Additionally, the penalty appears in all regions of the U.S., and reveals itself among many respondent subgroups — with especially large estimates among Republicans and lower income individuals. This effect is substantively important; consistent with an additive credential model, the penalty against Asians is equivalent to roughly one additional degree step (e.g., BA → MA) or about 1.3 additional years of work experience — placing a hidden tax that Asians must overcome. Pairing our conjoint with unique independent name-perception ratings shows that this discrimination is driven, in part, by perceptions of Asians as noncitizens or perpetual foreigners. Going one step further, we find that cultural-assimilation signals eliminate this gap. Anglicization of the first name closes it entirely: Asian candidates with Anglicized first names are selected by workers for upper management positions at parity with White, Black, and Hispanic candidates. Our results suggest that the modern bamboo ceiling reflects an actionable nameand credential-based cultural-integration filter rather than a generalized racial penalty.
- Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and Maya Lu. "The China Penalty: Measuring and Understanding a Blanket Bias Against Chinese Immigrants Across 22 Nations"
How are Chinese immigrants received in societies that increasingly view their homeland as a threat? Using a large-scale (N ≈ 47,000) conjoint experiment, that presents respondents with profiles of potential immigrants, across 22 of China's geopolitical rivals — spanning Europe, North America, Asia, Latin America, and Africa — we document a consistent and significant penalty against potential Chinese immigrants relative to potential immigrants from non-stigmatized countries. We systematically test each of the leading candidate mechanisms, which include ethnonationalism, chauvinism, and hostility toward foreign investment. None accounts for the penalty. The penalty is diffuse and pervasive, cutting across attitudinal subgroups regardless of respondents' nationalist, chauvinistic, or economic orientations. We further find that conventionally rewarded immigrant attributes such as high skills and host-country language fluency do not differentially benefit Chinese applicants: these premiums hold, but they raise the floor without closing the gap. The Chinese penalty is additive rather than conditional. Individual credentials cannot neutralize a country-of-origin stigma that prevailing theories of immigration attitudes only partially explain.
- Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and Surili Sheth. "Forever Foreign? Differential Linguistic Assimilation Standards for Asian Immigrants"
Do Americans hold immigrant groups to different assimilation standards? Using conjoint experiments with 2,999 Americans who evaluated which of 59,980 immigrant profiles was “most assimilated,” we find that linguistic assimilation matters significantly more for Asian immigrants than for other groups. Asian immigrants face a 20% larger penalty (Wave 1) to 65% larger penalty (Wave 2) for speaking their native language at home compared to White immigrants. However, cultural identification with America matters equally across all racial groups. Importantly, this differential is conditional: among Asian immigrants with fluent English, speaking a native language at home is not penalized and may even be valued, whereas those with limited English face severe penalties. These findings suggest that the “perpetual foreigner” stereotype operates through observable behavioral cues interpreted in light of English proficiency, rather than categorically or through internal identities. The results have implications for understanding immigrant incorporation, the boundaries of national identity, and the persistence of racial stereotypes.
- Gaddis, S. Michael, Edvard Nergård Larsen, Charles Crabtree, and John B. Holbein. "How Prevalent is Racial Discrimination Against Black and Hispanic Americans Across Contexts? A Meta-Analysis of Correspondence Audits."
To what extent does racial/ethnic discrimination in America differ across contexts? In this paper, we provide the largest and most comprehensive review of experimental research on racial/ethnic discrimination to date. We conducted a meta-analysis of 78 correspondence audits in the United States, representing over half a million applications, emails, and other forms of correspondence that occur in modern society, including the hiring, housing, medical, public services, and education sectors. Discrimination against racial/ethnic minorities exhibits substantial contextual heterogeneity not recognized in previous discrimination research. We find that racial/ethnic discrimination in the United States continues to be a large problem in some contexts but is less prevalent in other contexts. While discrimination against Black Americans is prevalent in hiring and rental housing, discrimination against Hispanic Americans is only prevalent in hiring. Although discrimination occurs in education, medical, and public services contexts, it is far less common in these sectors. Our findings suggest that discrimination is largest in contexts that are more resource-intensive and have higher short-term economic stakes, despite stronger legal protections against discrimination in those same contexts. Our work confirms that racial/ethnic discrimination in the United States continues to be a pervasive phenomenon that impacts many core parts of the lives of Black and Hispanic Americans and simultaneously reinforces and exacerbates existing inequalities. Our meta-analysis provides a roadmap for the key areas where additional experiments on discrimination are needed.
Applied Methods10 papers
- Charles Crabtree and John B. Holbein. "How Strong Is the Evidence for Peaceful Resistance? Not Very." Conditionally Accepted, American Political Science Review
Are peaceful social movements really more effective than violent ones? Chenoweth and Stephan's Why Civil Resistance Works argues that nonviolent resistance consistently outperforms violent resistance in achieving democratic outcomes. This conclusion has had enormous influence among academics, political actors, the media, and on-the-ground activists. We find that some results were misreported in ways that make the data appear to support civil resistance when they do not. Moreover, we find that the results are not robust to modest and reasonable changes in model specification, that they are sensitive to omitted variable bias, and that they rely on instrumental variable models with implausible assumptions that yield unstable estimates. Taken together, our replication and extension suggest that the available cross-national evidence provides no clear basis for the claim that nonviolent campaigns are substantially more effective than violent ones. This is not evidence that nonviolence backfires; it is evidence that the strong, unconditional advantage reported in Why Civil Resistance Works is not supported by the data. Our findings underscore the importance of careful methodological choices when drawing broad conclusions about how social movements achieve their goals.
- Hans Hassell and John B. Holbein. "Navigating Potential Pitfalls in Difference-in-Differences Designs: Reconciling Conflicting Findings on Mass Shootings’ Effect on Electoral Outcomes" American Political Science Review 119, no. 1 (2025): 240-260.
Work on the electoral effects of gun violence in the U.S. relying on difference-in-differences designs has produced findings ranging from null to substantively large effects. However, as difference-in-difference designs, on which this research relies, have exploded in popularity, scholars have documented several methodological issues including potential violations of parallel-trends and unaccounted for treatment effect heterogeneity. These pitfalls (and their solutions) have not been fully explored in political science. We apply these advancements to the unresolved debate on gun violence's effects on U.S. electoral outcomes. We show that studies finding a large positive effect of gun violence on Democratic vote shares are a product of a failure to properly specify difference-in-differences models when underlying assumptions are unlikely to hold. Once these biases are corrected, shootings show little evidence of sparking large electoral change. Our work clarifies an unresolved debate and provides a cautionary guide for scholars currently employing difference-in-differences designs.
- Mia Costa, Charles Crabtree, John B. Holbein, and Michelangelo Landgrave. "Is that Ethical? An Exploration of Political Scientists’ Views on Research Ethics" Research & Politics 10, no. 4 (2023): 20531680231209553.
What do political scientists think about research ethics? What research practices do they find acceptable? Using a survey conducted with the American Political Science Association, we explore perceptions of ethics among 362 political scientists. We find that political scientists do not place much relative weight on ethics when evaluating research. We do, however, find that researchers view different modes of inquiry as having different potential harms. Furthermore, using a conjoint experiment, we find that factors like author affiliation, study location, method, and sample size shape evaluations of study ethicality. Our results contribute to the growing body of metascience by expanding understanding of ethics in political science research.
- Crabtree, Charles D., S. Michael Gaddis, Cameron Guage, John B. Holbein, Jae Yeon Kim, and William W. Marx. "Validated Names for Experimental Studies on Ethnicity and Race" Nature: Scientific Data. Vol. 10, no. 130 (2023) Pick names with this paper’s data →
A large and fast-growing number of studies across the social sciences use experiments to better understand the role of race in human interactions, particularly in the American context. Researchers often use names to signal the race of individuals portrayed in these experiments. However, those names might also signal other attributes, such as socioeconomic status (e.g., education and income) and citizenship. If they do, researchers would benefit greatly from pre-tested names with data on perceptions of these attributes; such data would permit researchers to draw correct inferences about the causal effect of race in their experiments. In this paper, we provide the largest dataset of validated name perceptions to date based on three different surveys conducted in the United States. In total, our data include over 44,170 name evaluations from 4,026 respondents for 600 names. In addition to respondent perceptions of race, income, education, and citizenship from names, our data also include respondent characteristics. Our data will be broadly helpful for researchers conducting experiments on the manifold ways in which race shapes American life.
- Charles Crabtree, John B. Holbein, and Kiyoteru Tsutsui. "The Tyranny of Alphabetical Ordering: The Uneven Distribution of Authorship Credits by Authors' Country of Origin" Revised and Resubmitted at the Journal of Informetrics
Many scholars treat alphabetical ordering of authorship as a neutral convention for allocating credit in academic publishing in the social sciences. We show that this practice systematically advantages scholars from some countries while disadvantaging others, generating a hidden and persistent source of global inequality in academic recognition. Drawing on a century of publications from leading journals in economics, political science, and sociology, we document that scholarly surnames cluster around a median initial of K, creating a structural threshold that determines alphabetical advantage versus disadvantage under alphabetized bylines. We then combine these publication patterns with the most comprehensive global database of name usage to date — nearly half a billion name profiles across more than 100 countries — to characterize country-level surname-initial distributions. Our analysis reveals stark cross-national disparities: scholars from countries whose population surnames cluster later in the alphabet systematically appear later in bylines. These disadvantages cluster geographically, falling disproportionately on South America and East and Central Asian countries. Regression analyses suggest that cultural-linguistic factors, particularly script families, rather than economic development, better explain cross-country variation in surname-initial distributions. Our findings demonstrate that alphabetical ordering operates not as a neutral convention but as a structural mechanism that reproduces global inequalities in academic credit.
- Charles Crabtree and John B. Holbein. "Attention-Check Failure as a Survey-Level Diagnostic of Human Participation in Online Surveys" Rejection with an Invitation to Resubmit at Proceedings of the National Academy of Sciences
AI agents now pass attention checks at near-perfect rates, but the field’s most pressing data quality challenge is not AI contamination—it is the enormous variation in human respondent quality across survey platforms and the absence of any public data with which to benchmark it. We propose that providers publish aggregate attention-check passage rates at regular intervals. To show why, we formalize a mixture-model diagnostic exploiting the asymmetry that humans fail attention checks at nontrivial rates while AI agents do not, and apply it to published cross-platform data. The diagnostic fails as a cross-platform tool because human quality variation overwhelms any contamination signal; it succeeds only as a within-platform longitudinal monitor—which requires baselines no provider currently publishes. A correspondence with twelve survey providers confirms this gap. We call on providers to make baseline data public and on journals to require their disclosure.
- Charles Crabtree, John B. Holbein, Bart Bonikowski, and Kiyoteru Tsutsui. "Attention Checks Measure More Than Attention: Comparing Class Bias Across Survey Quality Indicators"
Attention checks are widely used across the social sciences to identify inattentive survey respondents, with researchers routinely excluding those who fail. Using data from 45,401 respondents across 22 countries, we demonstrate that attention check failure strongly correlates with socioeconomic status: excluding failers overrepresents collegeeducated respondents by 9.9 percentage points and higher-income respondents by 0.38 quintiles. Critically, we compare attention checks to an alternative quality indicator—satisficing detection via survey completion time—and find no class bias with the latter approach. This divergence suggests that attention checks conflate data quality with respondent resources, measuring comprehension and literacy as much as attentiveness. We recommend researchers report results stratified by attention check performance rather than excluding failers.
- John B. Holbein, Charles Crabtree, and Zander Furnas. "How Liberal Are Professors? What Existing Measures Can and Cannot Tell Us"
Every available measure of faculty political ideology is methodologically flawed. Surveys, social-media traces, voter files, and campaign contributions all observe selected behaviors, not ideology directly. Each channel conditions on a different form of participation. Response. Platform adoption. Eligibility. Disclosure. Donation. Record linkage. And each form of participation opens the door to external-validity problems, colliders, post-treatment selection, and measurement error. To be clear: the available evidence does point toward a faculty population that leans left of center, especially in the most politically visible groups. What it does not pin down is how far that pattern extends to the much larger share we do not observe, how precise any all-faculty estimate can be, or what would count as enough viewpoint diversity in the first place. We use DAGs, sensitivity analysis, and a selection-aware Bayesian meta-analysis to mark out what these measures actually support. The 2024 donor data observes 13.2 percent of faculty. Conclusions about the other 86.8 percent therefore rest on assumptions, not data. Headline estimates of the faculty left or Democratic share run from 38 percent in voter-registration data to 95.9 percent in 2026 donor data. A 58-point spread. Anchoring on the published literature narrows that gap, but only to 41 points. The Bayesian model puts the underlying issue in the open: under plausible directional selection-bias priors, the donor-channel posterior on all-faculty ideology moves by roughly 31 percentage points, nearly eight times the shift seen in higher-coverage voter-registration data. The bias assumption, not the data, does the inferential work. What we do not observe swamps what we do. We set out what the existing measures support and what they do not. We close with a reporting checklist for estimands, selection mechanisms, normative benchmarks, multi-measure ranges, and sensitivity analyses.
- John B. Holbein. "Do Natural Disasters Really Move Votes? Why the Evidence Does Not Survive Modern Difference-in-Differences Methods"
Natural disasters are a central proving ground for retrospective voting: if voters hold governments to account, disastrous events should show it. The evidence rests on a few foundational studies estimated before difference-in-differences research confronted the fragility of its core assumption, parallel trends. I revisit the two most influential—Gasper and Reeves (2011) and Healy and Malhotra (2009)—on their own data. Both reproduce. In both, however, pre-treatment trends do not run parallel: the outcomes of treated counties were already trending before any disaster arrived, and the headline effects—the disaster's and the response's alike—do not survive modest, formal allowances for those violations. Extending the test across a half-century of elections with damage and response measures yields the same null at the federal, state, and local levels. Voters may respond to disasters far more tepidly, if at all, than previously thought.
- John B. Holbein. "How Underpowered Are Close-Election Regression Discontinuity Designs? Evidence from U.S. Senate, House, Gubernatorial, State Legislative, and Local Races"
Close-election regression discontinuity (RD) designs are widely used given their promise of clean identification of the electoral effects of holding office. While facing scrutiny previously surrounding identification issues and the relevance of the estimands they produce, their statistical power has gone largely unexamined. Using contested U.S. Senate, House, gubernatorial, state legislative, and local election margins, this paper documents how severely underpowered close election regression discontinuity designs typically are. At the ±5-percentage-point bandwidth common in published work, Senate and gubernatorial designs reach under 10% power for a small effect (Cohen's d = 0.2) and the House 27%; even at ±10 pp the House reaches just 48%. In Congress the close races never accumulate fast enough: reaching 80% power would take the House over 150 years and the Senate more than a millennium. Pooled subnational designs are the exception, clearing conventional power across thousands of state legislative and local races; but that power belongs to the pool, and a single state or city, even over decades, falls back to single digits. Underpowered designs that do reach significance badly exaggerate effects and occasionally reverse their sign. A re-analysis of a prominent extremist-primaries RD makes the stakes concrete: its minimum detectable effect exceeds the largest effect the study reports. Scholars should exercise caution when leveraging this research design.
Education/Health7 papers
- Holbein, John B and Charles Crabtree. "Do Sleep Disruptions Promote Social Fragmentation?" Politics and the Life Sciences Vol. 42; no. 2 (2023): 205-233
Sleep changes predate shifts in mood/affect, thought processing, mental and physical health, civic engagement, and contextual circumstances, among other things. Theory predicts that these changes may lead to shifts in political and social beliefs. Do sleep disruptions shape how individuals see the world, the people around them, and themselves in relation to others? In this article, we use daily survey data from the 77 waves (N ≈ 460,000) of the University of California, Los Angeles’s 2019–2021 Nationscape Survey—a nationally representative political survey—to examine the effect of an exogenous short-term sleep disruption on measures of political views, polarization, and discriminatory beliefs. Using this data set, we leverage the modest sleep disruption that occurs at the start (and end) of Daylight Saving Time (DST) and employ a regression discontinuity in time design around the precise DST cutoff (which we supplement with event study models). Despite strong theoretical expectations and correlational connection between measures of sleep and many outcomes related to social fragmentation, we find that the DST change has little to no causal effect on citizens’ levels of polarization or their discriminatory attitudes. These effects are precise enough to rule out small effects, robust to a host of specification checks, and consistent across potential subgroups of interest. Our work adds to a small but growing body of research on the social and political effects of sleep disruptions.
- Crabtree, Charles D., John B. Holbein, and Quin Monson. "Patients' Traits Shape Healthcare Providers' Choices on How to Best Allocate Life-Saving Care" Nature Human Behaviour 6 (2022): 244-257
During global pandemics, health-care decision makers often face critical shortages of life-saving medical equipment. How do medical stakeholders prioritize which patients are most deserving of scarce treatment? We report the results of three conjoint experiments conducted in the United States in 2020, testing for biases in US physicians’, citizens’ and elected politicians’ preferences for scarce ventilator distribution. We found that all stakeholders prioritized younger patients and patients who had a higher probability of surviving with ventilator access. When patients’ survivability was tied, physicians prioritized patients from racial/ethnic minorities (that is, Asian, Black and Hispanic patients) over all-else-equal white patients, religious minorities (that is, Muslim patients) over religious majority group members (that is, Catholic patients) and patients of lower socio-economic status over wealthier patients. The public also prioritized Black and Hispanic patients over white patients but were biased against religious minorities (that is, Atheist and Muslim patients) relative to religious majority group members. Elected politicians were also biased against Atheist patients. Our effects varied by political party—with Republican physicians, politicians and members of the public showing bias against religious minority patients and Democratic physicians showing preferential treatment of racial and religious minorities. Our results suggest that health-care stakeholders’ personal biases impact decisions on who deserves life-saving medical equipment.
- Holbein, John B., Jerome Schafer, and David Dickinson. "Insufficient Sleep Reduces Voting and Other Prosocial Behaviors." Nature Human Behaviour 3, May (2019): 492-500. [Impact Factor: 24.25]
Insufficient sleep is a growing public health concern in industrial societies. Although a lack of sleep is known to negatively affect private behaviours—such as working or going to school—comparatively little is known about its consequences for the social behaviours that hold society and democracy together. Using three complementary methods, we show how insufficient sleep affects various measures of civic participation. With survey data from two countries, we show that insufficient sleep predicts lower voter turnout. Next, with a geographical regression discontinuity design, we demonstrate that individuals from the United States who tend to sleep less due to circadian impacts of time-zone boundaries are also less likely to vote. Finally, we experimentally manipulate short-term sleep over a two-stage study. We observe that the treatment decreases the levels of civic engagement, as shown by their willingness to vote, sign petitions and donate to charities. These results highlight the strong negative consequences that current levels of insufficient sleep have on vitally important measures of social capital.
- Holbein, John B. & Helen F. Ladd. “Accountability pressure: Regression discontinuity estimates of how No Child Left Behind influenced student behavior.” Economics of Education Review 58 (2017): 55-67
In this paper we examine how failing to make adequate yearly progress under No Child Left Behind (NCLB), and the accountability pressure that ensues, affects various non-achievement student behaviors. Using administrative data from North Carolina and leveraging a discontinuity in the determination of school failure, we examine the causal impact of this form of accountability pressure both on student behaviors that are incentivized by NCLB and on those that are not. We find evidence that, as NCLB intends, pressure encourages students to show up at school and to do so on time. Accountability pressure also appears to have the unintended effect, however, of increasing the number of student misbehaviors. Further, we find some evidence that this negative response is most pronounced among minorities and low performing students: those who are the most likely to be left behind.
- Ladd, Helen F., Charles T. Clotfelter, and John B. Holbein. “The Growing Segmentation of the Charter School Market in North Carolina” Education Finance and Policy 12, no. 4 (2017): 536-563.
A defining characteristic of charter schools is that they introduce a strong market element into public education. In this paper, we examine through the lens of a market model the evolution of the charter school sector in North Carolina between 1999 and 2012. We examine trends in the mix of students enrolled in charter schools, the racial imbalance of charter schools, patterns in student match quality by schools' racial composition, and the distributions of test score performance gains compared to those in traditional public schools. In addition, we use student fixed effects models to examine plausibly causal measures of charter school effectiveness. Our findings indicate that charter schools in North Carolina are increasingly serving the interests of relatively able white students in racially imbalanced schools and that despite improvements in the charter school sector over time, charter schools are still no more effective on average than traditional public schools.
- Charles Crabtree, Will Elliott, John B. Holbein, Devontae Lacasse, and Jayanth Uppaluri. "What Does the Public Really Want in College Admissions?" Invitation to Revise and Resubmit at Nature Human Behaviour
What types of students should be admitted to prestigious colleges according to the American public? We conducted a large conjoint experiment with a sample of 2,580 U.S. adults; respondents evaluated a total of 25,800 hypothetical applicants. We randomly varied the applicants’ race, academic credentials, socioeconomic background, country of origin, and disability status. We find that academic credentials dominate public preferences: the public strongly prefers college applicants with higher GPAs, stronger SAT scores, and/or evidence of leadership in athletics or extracurricular activities. By contrast, race has only modest effects; in their actions within the conjoint, citizens indicate a preference that racial groups be treated comparatively similarly in admissions decisions. Though there are some differences by respondents’ own race/ethnicity—with Black and Hispanic individuals favoring their groups being, all-else-equal, given preferential treatment—these differences are small relative to other features. In contrast, there is some evidence that the public believes that college admissions should take into account the difficulties faced by socioeconomically disadvantaged individuals—indeed, our experimental results show that the public prefers low-income applicants to receive sizable advantages over high-income applicants. Simultaneously, however, the public prefers that students from the United States receive preferential admissions over immigrants. Finally, we show that the largest penalties arise from mental health disclosure: psychiatric conditions reduce applicants’ selection probability by a substantial amount.
- Charles Crabtree, John B. Holbein, Cecilia Hyunjung Mo, and Surili Sheth. "Do Virtual Museums Highlighting the Experiences of Minorities Persuade Visitors? Evidence From a Study on Bias Toward Asian Americans" Under Review
Biases against individuals of Asian descent persist in the United States. Can cultural history museums, which aim to provide public education on the struggles, achievements, and cultural traditions of different groups, reduce these biases? We address this question through an experiment using virtual museum content curated by the Museum of Chinese in America (MOCA). By embedding MOCA’s virtual exhibits into surveys, we test whether exposure to the lived experiences and history of the Chinese diaspora in the United States alters non-Asian Americans’ attitudes toward and knowledge about Asian Americans. Our findings suggest that historical and emotionally resonant messaging common in museum exhibits improve attitudes and beliefs, primarily among individuals who already hold positive racial predispositions. These results highlight both the promise and limits of museum-based interventions aimed at reducing out-group bias, and speak to the efficacy of scalable digital interventions.