Library · Causal inference & statistics
What Difference-in-Differences Design got wrong, from 48 theses
Summary written by a model from the records below: The records document methodological obstacles and rejection criteria encountered when applying difference-in-differences designs across empirical evaluations. Most methodological failures stem from violations of core identifying assumptions such as parallel pre-trends, staggered adoption biases, unobserved confounding, and spillover effects. These records come from graduate theses, 2021 to 2026, each labeled with its degree where the repository states it, and each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Plus 9 problems the authors left open, listed at the end.
Failure modes, by theses reporting them
Select a mode to jump to its records.
Pre-treatment parallel trends assumptions were violated across treated and control groups
Empirical analyses frequently failed pre-intervention trend tests, placebo falsification checks, and event-study diagnostics, preventing valid causal attribution. Researchers rejected standard designs after detecting pre-existing divergence between cohorts, baseline macroeconomic shocks, and severe estimate instability from modest trend deviations.
Tried and failed
difference-in-differences policy evaluation applied to subgroup employment after policy intervention. Reason: parallel trends assumption was violated in pre-treatment periods, preventing causal identification
The Impact of Medicaid Policy Change on People with Disabilities · Harvard · PhD
Tried and failed
doubly robust difference-in-differences estimator applied to regional income inequality aggregate indices. Outcome: no signal. Reason: violation of parallel-trend assumptions and no statistically significant effect on aggregate inequality metrics
Essays on income inequality in the United States · UT Austin · PhD
Tried and failed
stacked difference-in-differences with synthetic controls applied to healthcare labor policy impact evaluation. Reason: pre-period parallel trends assumption failed statistical significance test in low baseline states
The Impact of Medicaid Policy Change on People with Disabilities · Harvard · PhD
Tried and failed
difference-in-differences across income eligibility threshold applied to earned income tax credit effects. Reason: severe violations of parallel pre-trends between treatment and control groups near threshold
Essays in Public Economics and Labor Economics · Harvard · PhD
Tried and failed
standard difference-in-differences estimation applied to state income tax reform impacts. Reason: Severe pre-reform downward trends in income and population violated the parallel trends assumption.
ESSAYS IN APPLIED MICROECONOMICS · Cornell · PhD
Tried and failed
difference-in-differences with two-way fixed effects applied to evaluating policy effects on crime rates. Reason: violation of parallel trends assumption demonstrated by failed pre-trend placebo tests
Three essays on economics of crime and immigration · Texas Tech · PhD
Tried and failed
difference-in-differences estimation applied to heterogeneous firm productivity and investment. Reason: pre-treatment trends differed significantly across firm productivity, capital, and investment levels, violating parallel trends
Essays in Applied Economics: Capital Allocation, Transmission of Global Uncertainty and Climate Mitigation · Cornell · PhD
Tried and failed
difference-in-differences estimation applied to retail experiments. Outcome: no signal. Reason: violation of the parallel trends assumption in over 90% of experiments
Learning and Optimization in Modern Retail · MIT · PhD
Tried and failed
Staggered difference-in-differences design applied to Policy impact on educational outcomes. Reason: Significant pre-existing trends violated the parallel trends assumption across treated and control units.
ESSAYS ON THE HUMAN CAPITAL AND OCCUPATIONAL CHOICES OF ADOLESCENTS · Cornell · PhD
Tried and failed
trimming sample outliers in difference-in-differences analysis applied to panel data with heterogeneous treatment. Reason: removing extreme units violated the parallel trends assumption required for valid causal inference
Coercive and catalytic strategies for promoting human rights : state violence and the composition of foreign aid · UT Austin · PhD
Tried and failed
event study difference-in-differences applied to maternal labor outcomes after childbirth. Reason: pre-existing differential trends violated the parallel trends assumption required for causal identification
Three Essays in Development Economics · Harvard · PhD
Tried and failed
difference-in-differences event study applied to aggregate mortality rates after policy expansion. Reason: pre-treatment divergence violated the parallel trends assumption required for causal inference
Essays in health economics · UT Austin · PhD
Tried and failed
difference-in-differences regression applied to heterogeneous staggered rollout panel data. Reason: Divergent pre-intervention trends and heterogeneous adoption timelines violated standard parallel trends assumptions.
Countries on the Edge: Evaluating the Sustainability of Gavi’s Immunization Support in Former Gavi-supported Countries: Evidence from the COVID-19 Pandemic · Harvard · Doctorate
Considered and rejected
Considered and rejected: Rejected standard Difference-in-Differences (DID) due to violation of the parallel trends assumption between state cohorts.
Essays in applied economics · Texas Tech · PhD
Considered and rejected
Considered and rejected: Difference-in-Differences (DiD) was rejected for primary causal claims regarding parental labor market participation due to failure of the parallel pre-treatment trends assumption for maternal outcomes.
Socioeconomic implications of adverse birth outcomes · Leibniz Universität Hannover · PhD
Considered and rejected
Considered and rejected: Rejected standard Difference-in-Differences (DID) design for evaluating universal FAFSA policies because parallel trends do not hold across states.
Essays in the Economics of Education · Harvard · PhD
Considered and rejected
Considered and rejected: Traditional Difference-in-Differences methodology rejected due to violation of parallel trends from staggered rollout and COVID-19 moratorium shocks.
Essays in applied microeconomics · Boston University · PhD
Considered and rejected
Considered and rejected: Rejected difference-in-differences design because non-random DP signings make violation of the parallel trends assumption highly likely
Considered and rejected
Considered and rejected: Standard two-way fixed effects difference-in-differences (DiD) was rejected because ban and non-ban states violated the parallel trends assumption prior to Dobbs and staggered adoption causes bias.
The Effects of State Abortion Bans on Reproductive Health and Mortality: A Bayesian Hierarchical Low-Rank Difference-in-Differences Approach · Harvard · PhD
Considered and rejected
Considered and rejected: Naive difference-in-differences without propensity score adjustment was rejected due to severe parallel trends violations from the Great Recession
Essays in Labor and Public Economics · Cornell · PhD
Considered and rejected
Considered and rejected: Rejected difference-in-differences comparing corequisite and non-corequisite colleges across the TSI distribution due to failure of the parallel trends assumption outside the eligibility window.
Essays on inference and education economics · UT Austin · PhD
Tried and failed
difference-in-differences policy evaluation applied to health IT adoption and hospital outcomes. Outcome: unstable. Reason: initially significant effect estimates vanished under modest violations of parallel trends assumptions
From Theory to Practice: Improving Causal Conclusions from Healthcare Data · MIT · PhD
Tried and failed
stacked event study difference-in-differences applied to evaluating policy impacts on mortality rates. Reason: pre-intervention trends between treated and control groups failed the parallel trends assumption
Pressing Challenges in U.S. Health Policy: Polarization, Gun Violence, And Discrimination · Harvard · PhD
Staggered rollout and two-way fixed effects produced biased comparisons with negative weights
Conventional two-way fixed effects estimators generated substantial econometric bias by using already-treated cohorts as controls in settings with dynamic treatment effects. Authors rejected standard staggered models or found that estimated effects disappeared after replacing them with robust estimators or stacked cohort designs.
Tried and failed
staggered difference-in-differences with robust estimators applied to evaluating corporate investment policy impacts. Outcome: no signal. Reason: treatment effects disappeared after correcting for staggered adoption bias and outlier winsorization
Essays on the Impact and Effectiveness of Share Repurchase Regulations · Harvard · PhD
Considered and rejected
Considered and rejected: Rejected canonical Difference-in-Differences models with period and group fixed effects due to biased estimates arising from 'forbidden' comparisons of already-treated units under heterogeneous treatment effects.
Beyond Trade Losses: Sanctions and the Resilience of Productive Economies · Texas Tech · PhD
Considered and rejected
Considered and rejected: Rejected standard two-way fixed effects (TWFE) difference-in-differences due to treatment effect heterogeneity across adoption cohorts and negative weighting issues.
Essays on applied labour economics: personality traits, artificial intelligence and formalisation policies · Nottingham · PhD
Tried and failed
two-way fixed effects difference-in-differences estimation applied to staggered policy adoption evaluation. Reason: pervasive negative weights on early-treated cohorts and failure of treatment homogeneity assumptions
Three Essays on the Economics of Agricultural Production Behavior, Renewable Natural Resources, and Welfare Dynamics · Cornell · PhD
Considered and rejected
Considered and rejected: Rejected standard Two-Way Fixed Effects (TWFE) with staggered treatment due to time-varying treatment effects leading to biased comparisons with already-treated units, adopting stacked difference-in-differences instead
Essays on infrastructure and urban development in developing countries · Boston University · PhD
Considered and rejected
Considered and rejected: Decided against using traditional Difference-in-Differences estimators because they average exposure-outcome associations over time and can be biased when associations vary as policy generosities change.
Structural Responses to Structural Inequities: Evaluating the Potential of Earned Income Tax Credit (EITC) Policies as Tools for Addressing Social Inequities in Mental Distress · University of Washington · PhD
Considered and rejected
Considered and rejected: Rejected standard two-way fixed-effects (TWFE) estimators in favor of stacked difference-in-differences to avoid bias from heterogeneous/staggered treatment timing.
Impacts of State Abortion Restrictions on Mental Health and Healthcare · Johns Hopkins · PhD
Considered and rejected
Considered and rejected: Rejected staggered difference-in-differences as the primary empirical design due to econometric concerns regarding staggered DiD estimates, relegating it to a robustness check.
Do Private Tax Disclosures Affect the Quality of Public Financial Reporting? · University of Oregon · PhD
Considered and rejected
Considered and rejected: Rejected staggered difference-in-differences designs due to econometric bias from heterogeneous treatment timing, opting for canonical 2x2 DD models
Manifestations of the Positive Death Movement in America: Medical Aid In Dying, Voluntarily Stopping Eating & Drinking, and End-Of-Life Doulas · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Rejected standard two-way fixed-effects (TWFE) difference-in-differences models due to bias caused by dynamic/staggered treatment timing and negative weights comparing early-treated to late-treated units.
Three Essays On Economics of Marriage Law · Texas Tech · PhD
Considered and rejected
Considered and rejected: Rejected standard Two-Way Fixed Effects (TWFE) staggered difference-in-differences due to contamination and bias from heterogeneous cohort treatment effects, adopting the Wooldridge (2021) two-way Mundlak regression instead.
Essays on Foreign Direct Investment and International Trade · University College Dublin · PhD
Omitted time-varying confounders and selection bias distorted treatment effect estimates
Models lacking adequate covariate adjustments, fixed effects, or selection controls produced counterintuitive coefficient signs and spurious long-term negative estimates. Unmeasured environmental shocks, concurrent disruptions, anticipatory behavior, and non-absorbing treatment states severely compromised causal identification.
Considered and rejected
Considered and rejected: Rejected using unweighted difference-in-differences as the sole model without testing multiple group propensity score weighting to address baseline covariate imbalance.
Indirect Effects of Financial Incentives on Physician Behavior in Perinatal Care · Johns Hopkins · PhD
Tried and failed
staggered difference-in-differences with robust estimators applied to longitudinal policy evaluation of student performance. Reason: produced spurious long-term negative effects likely driven by unobserved time-varying confounders rather than treatment
Art-tendance: The Effect of Creative Learning on Student Outcomes · UT Austin
Tried and failed
difference-in-differences without covariate adjustment applied to continuous spatial policy spillover estimation. Reason: unadjusted confounding produced counterintuitive positive treatment effect estimates at low dose ranges
Causal Inference Methods To Evaluate Health Policies With Spillover · Penn · PhD
Tried and failed
staggered difference-in-differences using localized shock applied to environmental quality impact on student performance. Outcome: no signal. Reason: Confounding negative effects from psychological trauma and commuting disruptions masked any environmental benefits.
Three Essays in Applied Microeconomics · Georgia Tech · PhD
Tried and failed
difference-in-differences on panel data applied to international trade policy interventions. Reason: unmeasured confounding and anticipatory effects yielded counterintuitive negative causal estimates
Identification and Estimation of Policy-relevant Causal Effects from Observational Data · EPFL · PhD
Considered and rejected
Considered and rejected: Rejected relying solely on OLS and Difference-in-Differences with inconsequential units approach due to non-random intermediate small cities and time-varying omitted environmental confounders (e.g., pollution, noise).
Essays on Health and Transportation Economics · University at Buffalo · PhD
Tried and failed
matched difference-in-differences estimation applied to school-level education intervention effects. Reason: unobserved group-by-cohort selection bias caused spurious negative point estimates
Building Communities of Upward Mobility · Harvard · PhD
Considered and rejected
Considered and rejected: Rejected relying on standard matched difference-in-differences estimators because treatment was not strictly an absorbing state across cohorts and schools suffered negative selection shocks.
Building Communities of Upward Mobility · Harvard · PhD
Bounded outcomes, variable scaling, and aggregation obscured or misspecified treatment effects
Applying linear difference-in-differences to discrete or bounded measures generated out-of-bounds counterfactual probabilities and conflated treatment with mean reversion. In other settings, pooling distinct locations obscured localized impacts, or treatment effects disappeared under alternative variable scaling and normalization choices.
Considered and rejected
Considered and rejected: Rejected standard Difference-in-Differences (DiD) / parallel trends for discrete and bounded outcomes because it confounds baseline mean reversion with treatment effects and can produce counterfactual probabilities outside the [0, 1] range
PROGRAM EVALUATION OF TREATMENT EFFECT HETEROGENEITY: THEORY AND APPLICATIONS · Penn · PhD
Tried and failed
Difference-in-differences with alternative variable scaling applied to Corporate expenditure and investment ratios. Outcome: no signal. Reason: Treatment effects lost statistical significance when scaling expenditures by revenue instead of lagged total assets
Considered and rejected
Considered and rejected: Rejected Difference-in-Differences (DiD) because it constrains predictor effects to be constant over time rather than accounting for weighted variable changes.
A Sequential Explanatory Mixed Methods Study Measuring the Effects of Implementing the Bayh-Dole in International Contexts · Texas Tech · PhD
Tried and failed
pooled difference-in-differences estimator applied to downstream property price changes. Outcome: no signal. Reason: pooling all sites obscured heterogeneous treatment effects across locations
Essays on environmental economics · UT Austin · PhD
Interventions violated SUTVA or lacked valid untreated comparison groups
Spatial spillovers and broad regulatory scope violated stable unit treatment value assumptions across market boundaries. In addition, universal policy implementations and pervasive rollouts left no clean, contemporaneous untreated units to serve as valid control groups.
Considered and rejected
Considered and rejected: Rejected standard Difference-in-Differences for evaluating the 2017 German corporate tax reform because all German corporate startups were simultaneously affected, adopting Synthetic Control instead.
Essays on the Finance of Startups · Tübingen · PhD
Considered and rejected
Considered and rejected: Rejected difference-in-differences/regression discontinuity due to lack of a clean exogenous shock affecting only MAIs without impacting all patenting firms.
The value of analytics innovations · Iowa State · PhD
Considered and rejected
Considered and rejected: Rejected standard Difference-in-Differences (DiD) in favor of pooled OLS / SAR because the SUTVA assumption failed due to spatial spillovers and shifting net-exporter status.
Spatial Dimensions of Economic Modeling: Interdisciplinary Approaches to Labor, Trade, and Networks · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Standard difference-in-differences design between colonias with and without service was rejected due to small sample sizes and gradual rollout across all counties.
Municipal infrastructure and public policy : program evaluation of three case studies along the U.S.-Mexico border · UT Austin · PhD
Left open by the authors
Problems the authors named and did not get to.
Left open
Estimate the causal effect of continuous per-dollar minimum wage increases on health outcomes using continuous-treatment difference-in-differences methods. Blocker: None
Impact of Social Policies on Health · Harvard · PhD
Left open
Modify difference-in-differences minimum wage empirical designs to incorporate cross-state worker migration leakages. Blocker: None
How the Price System Works: Evidence from Supply Chains and Price Controls · Harvard · PhD
Left open
Estimate difference-in-differences regressions interacting minimum wage increases with PNTR tariff exposure shocks to measure contemporaneous buffering effects on local crime rates. Blocker: None
Three Essays on the Impacts of Trade Liberalization · Georgia Tech · PhD
Left open
Estimate compulsory schooling law impacts on child labor, marriage age, bride prices, sibling allocations, and birth spacing using difference-in-differences. Blocker: None
Essays on China’s Economic Development · Cornell · PhD
Left open
Test parallel trends assumptions in ACA Medicaid expansion difference-in-differences models using linked individual microdata rather than aggregate mortality data. Blocker: Requires access to restricted administrative individual microdata (e.g., linked Census/vital statistics records).
Essays in health economics · UT Austin · PhD
Left open
Estimate the causal effect of carbon pricing on provincial industrial greenhouse gas emissions and emissions intensity using staggered Difference-in-Differences. Blocker: None
The causal effect of carbon pricing on industrial energy consumption in Canada · University of Manitoba · Master's
Left open
Perform difference-in-differences analysis testing parallel trends and health trajectory selection on subsequent IFLS survey waves. Blocker: None
The Hidden Costs of Mobility: Changing Patterns of Labor Migration and Health Implications for Left Behind Families in Indonesia. · Cornell · PhD
Left open
Evaluate longer-term earnings outcomes beyond age 20 using matched difference-in-differences on longitudinal administrative wage and education records. Blocker: Requires access to restricted administrative longitudinal earnings data linked to student records.
Essays on health economics and public policy · UT Austin · PhD
Left open
Evaluate whether Science Based Targets participation affects downside beta, drawdowns, and return volatility using matched pair difference-in-differences regression on S&P 500 firms. Blocker: None
Innovation Through a Net Zero Economy and the Impact to an Investors Bottom Line · Harvard · Master's
Who has worked on this
Authors of the public theses cited on this page, most-cited first.
- Federico Gonzalez · Harvard · PhD · 2025
Building Communities of Upward Mobility - Ari Ne'eman · Harvard · PhD · 2024
The Impact of Medicaid Policy Change on People with Disabilities - Matlin Gilman · Harvard · PhD · 2026
The Effects of State Abortion Bans on Reproductive Health and Mortality: A Bayesian Hierarchical Low-Rank Difference-in-Differences Approach - Patricia Carrington · Harvard · Master's · 2026
Innovation Through a Net Zero Economy and the Impact to an Investors Bottom Line - Rimvydas Ragauskas · Texas Tech · PhD · 2026
Beyond Trade Losses: Sanctions and the Resilience of Productive Economies - Matias Alejandro Golman · Nottingham · PhD · 2026
Essays on applied labour economics: personality traits, artificial intelligence and formalisation policies - Chiyoung Ahn · Penn · PhD · 2026
PROGRAM EVALUATION OF TREATMENT EFFECT HETEROGENEITY: THEORY AND APPLICATIONS - Amirali Kamalirad · University of Manitoba · Master's · 2026
The causal effect of carbon pricing on industrial energy consumption in Canada - Sharan Banerjee · Cornell · PhD · 2025
Essays in Applied Economics: Capital Allocation, Transmission of Global Uncertainty and Climate Mitigation - Yuting Wang · MIT · PhD · 2025
The Causal Effects of Mandatory Quarterly Earnings Guidance on Corporate Information Environment and Corporate Short-Termism - Emtiaz Hossain Hritan · Georgia Tech · PhD · 2025
Three Essays in Applied Microeconomics - Hyung Joon Kim · Harvard · Doctorate · 2025
Countries on the Edge: Evaluating the Sustainability of Gavi’s Immunization Support in Former Gavi-supported Countries: Evidence from the COVID-19 Pandemic
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.