JH John B. Holbein

Research Tools

Interactive tools for researchers.

Open-source tools built around datasets and methods from the lab. Designed for study design, data exploration, and teaching.

Experimental Design Power Analysis

Audit Study Power Calculator

Calculate the sample size needed to reliably detect racial discrimination. Supports within-subjects / paired designs (McNemar's test) and between-subjects / unpaired designs (two-proportion z-test). Adjust the control callback rate, effect size, within-pair correlation (ρ), significance level, and power — see a live power curve and side-by-side job-posting comparison. Use this before the Name Picker.

Formulas from Fleiss, Levin & Paik (2003) and Schlesselman (1982), verified against Neumark (2012, Journal of Human Resources). The ρ slider captures the within-firm callback correlation — positive values (realistic for hiring audits: 0.3–0.5) further reduce the required N for the paired design.

Launch tool → Neumark (2012)
Experimental Design Racial Discrimination

Validated Name Picker

Choose names for studies of racial discrimination — correspondence audits, conjoint experiments, vignettes, and survey experiments. Pick your racial comparison, optimization strategy, and sample size; see a balance diagnostic for every name selected.

Built from Crabtree, Gaddis, Guage, Holbein, Kim & Marx (2023), Nature: Scientific Data — the largest validated name dataset to date, covering 600 names and 44,170 evaluations across race, income, education, and citizenship perceptions.

Launch tool → Read the paper
Statistical Inference Power Analysis

Equivalence Testing Calculator

A null result is not the same as evidence of no effect. This tool implements two one-sided tests (TOST) to determine whether your result rules out effects large enough to matter — and how large a study you'd need to do so convincingly. Enter your observed effect, standard error, and equivalence bound (±Δ); see the verdict, an interactive diagram of your confidence interval against the bounds, and a required-N table across power levels.

Implements Schuirmann (1987, Journal of Pharmacokinetics and Biopharmaceutics) with power formulas and SESOI guidance from Lakens (2017, Social Psychological and Personality Science). Equivalence is established when the (1−2α)×100% CI falls entirely within ±Δ.

Launch tool → Lakens (2017)
Research Design Power Analysis

Close-Election RD Power Calculator

Most close-election regression discontinuity studies are severely underpowered. This tool computes exact power for U.S. Senate and House RD designs using real election margins — showing how many races fall within your bandwidth, what effect sizes you can realistically detect, and how long it would take at historical rates to accumulate enough close races for adequate power.

Uses 793 Senate races (1976–2024) and 9,348 House races (1976–2024) from the MIT Election Data Science Lab. Reproduces the core finding of Stommes, Aronow & Sävje (2023, Research & Politics): only 22% of published close-election RD studies achieve 80% power for a small standardized effect.

Launch tool → Stommes et al. (2023)