Library · Causal inference & statistics
What Multilevel & Hierarchical Linear Modeling got wrong, from 41 theses
Summary written by a model from the records below: The records document various practical failures and reasons for rejecting multilevel and hierarchical linear models in empirical research. Authors frequently encountered numerical non-convergence, negligible clustering variance, insufficient cluster counts, and violations of hierarchical nesting assumptions. These records come from graduate theses, 2021 to 2026, each labeled with its degree where the repository states it, and each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Plus 8 problems the authors left open, listed at the end.
Failure modes, by theses reporting them
Select a mode to jump to its records.
Negligible cluster variance and low intraclass correlations render multilevel models unnecessary
Studies frequently found intraclass correlation coefficients near zero or well below conventional thresholds such as five percent across groupings like sites, hospitals, and schools. Because clustering adjustments failed to alter regression estimates or explain outcome variance, researchers rejected multilevel frameworks in favor of standard single-level regressions.
Considered and rejected
Considered and rejected: Rejected 3-level and 2-level hierarchical/multilevel modeling in favor of linear regression with dummy-coded site predictors due to minimal cluster variance (ICC = .003 for parent empowerment, .021 for outpatient service use, 0 for child symptoms) and single-participant courses.
Moderators of treatment effects for a peer-led intervention for caregivers of children with mental health needs · UT Austin · PhD
Tried and failed
multilevel random intercept modeling with cluster-robust variance applied to hierarchical patient transfer outcome prediction. Reason: clustering adjustments did not meaningfully alter regression estimates compared to standard logistic regression
On healthcare self-organisation: a complexity approach to exploring patient journeys and outcomes · Imperial · PhD
Tried and failed
hierarchical linear modeling with random effects applied to hierarchical institutional and course data. Outcome: no signal. Reason: intraclass correlation coefficients were below the 5% threshold, rendering multilevel modeling unnecessary
Course withdrawals and college student success · UT Austin · PhD
Considered and rejected
Considered and rejected: Rejected multilevel hierarchical logistic regression for pandemic phase due to lack of level-2 clustering (ICC = 0.0018) and small number of clusters.
Immigrants’ Use of Online Mental Health Services during the COVID-19 Pandemic · Queen's University · Master's
Considered and rejected
Considered and rejected: Decided against multilevel regression for Aim 3 because the neighborhood intraclass correlation coefficient was virtually zero (ICC = 0.004), adopting single-level models instead.
HOUSING DISCRIMINATION AND HEALTH: EXPLORING THE RELATIONSHIP BETWEEN STRUCTURAL RACISM AND MENTAL HEALTH AMONG BLACK AMERICAN ADULTS · Johns Hopkins · PhD
Considered and rejected
Considered and rejected: Multilevel modeling rejected in favor of single-level path analysis due to non-convergence and low sibling clustering dependency (ICC = 0.013).
Likes, Looks, and Learning: Associations of Adolescent Engagement in Disordered Eating with Parent, Sibling, Peer, and Social Media Influences · Texas Tech · Master's
Considered and rejected
Considered and rejected: Rejected analyzing PSC via multilevel modelling (MLM) at group level due to limited clusters (two hospitals) and ICC variance <5%, opting for individual-level SEM.
The role of healthcare accreditation in influencing burnout and work engagement levels among healthcare professionals in The United Arab Emirates · Nottingham · Doctorate
Considered and rejected
Considered and rejected: Rejected multilevel modeling for Group 2 nested data because the number of clusters (4 genetic counselors) was too few and ICC was too small (0.03)
MODELING THE EFFECTS OF SURVEY TIMING ON SCORES FOR THE WORKING ALLIANCE INVENTORY IN A SECONDARY DATA ANALYSIS OF CARDIOVASCULAR GENETIC COUNSELING OUTCOMES · Johns Hopkins · Master's
Considered and rejected
Considered and rejected: Rejected mixed-effects logistic regression with site as a random intercept in favor of standard logistic regression due to negligible clustering (ICC ~ 0)
Considered and rejected
Considered and rejected: Multilevel modeling (hierarchical linear modeling) was rejected due to having only 35 school clusters (below the recommended 50) and non-significant ICCs (p = 0.26 and p = 0.13), reverting to OLS regression.
Supporting Youth Mental Health in Schools: An Investigation of Teachers as Gateway Providers · University at Buffalo · PhD
Considered and rejected
Considered and rejected: Multilevel modelling (using R lme4) was rejected due to model convergence failure on EDI Dysphoria, very low ICCs (2.6%–5%), and design effects < 2.
Therapist Factors on Autistic Children’s Socio-emotional Outcomes After Participating in a Cognitive Behavioural Intervention · Carleton University · Master's
Considered and rejected
Considered and rejected: Rejected multilevel factor analysis despite nested data (3 targets per participant) because intraclass correlation coefficients were very low (ICCs 0 to .27) and single-level analysis yielded identical factor structures.
Racialized and Multifaceted: Impacts of the Feminist Label and Race/Ethnicity on Perceptions of Men’s Masculinity · Queen's University · PhD
Considered and rejected
Considered and rejected: Rejected multilevel modeling with random intercepts due to very low ICCs (< 0.10) and model convergence/fitting failures, adopting standard logistic, negative binomial, and OLS regression with divisional fixed controls instead.
Investigating U.S. State-Level Income Inequality as a Determinant of Population Health: Theory, Evidence, and Directions Forward · University at Buffalo · PhD
Considered and rejected
Considered and rejected: Rejected multilevel structural equation modeling (ML-SEM/HLM) because school-level intraclass correlation coefficients (ICCs) ranged from .0015 to .0222, well below the .05 threshold.
Malleable mental health factors in undergraduate engineering students · UT Austin · PhD
Numerical non-convergence and optimization errors prevent model estimation
Multilevel models frequently failed to converge when fit to complex longitudinal trajectories, multi-trait outcomes, or nonlinear count data. Estimation failures also occurred when Gauss-Hermite numerical integration could not handle continuous moderators or when multi-predictor specifications produced singular fits.
Tried and failed
multilevel logistic regression with competing event predictor applied to hierarchical student transfer outcome modeling. Outcome: did not converge. Reason: linear dependency and data collinearity among predictors causing estimation failure
Which policies and practices influence vertical transfer and baccalaureate attainment among community college entrants? : a multi-level analysis · UT Austin · PhD
Tried and failed
Linear mixed-effects modeling applied to longitudinal plant growth count data. Outcome: did not converge. Reason: Singular fit errors caused by non-linearity in the response variable
Ecology and evolution of the African ant acacia, Vachellia drepanolobium, and its multiple symbionts · Harvard · PhD
Tried and failed
multilevel modeling with aggregate contextual predictors applied to longitudinal adolescent mental health data. Outcome: did not converge. Reason: insufficient convergence to compute higher-level standard errors
A life course exploration of social and temporal variability in the association between parental divorce and adolescent mental health trajectories · UT Austin · PhD
Tried and failed
linear mixed-effects models with random slopes applied to pooled multi-cohort longitudinal trajectory data. Outcome: did not converge. Reason: None
Accounting for Heterogeneity in Health Decision Analysis · Harvard · PhD
Lost to a baseline
In Study II weight interpolation with dropping third-trimester weights, the cubic polynomial mixed-effects model failed to converge.
Lifestyle, Weight Gain, and Pregnancy Complications from a Life-Course Perspective · Harvard · PhD
Considered and rejected
Considered and rejected: Simultaneous multi-trait multilevel regression predicting goal progress rejected due to non-convergence
Personality-Goal Fit: Benefits and Consequences of Aligning Personality Traits and Personal Goals · Carleton University · Master's
Considered and rejected
Considered and rejected: Rejected entering multiple PBAT items into a single multilevel model due to lack of model convergence
Examining the temporal dynamics of psychological flexibility on affect and stress in a transdiagnostic clinical sample: an ecological momentary assessment study · Boston University · PhD
Considered and rejected
Considered and rejected: Rejected full multivariable adjustment in multilevel variance estimation models due to convergence failure, retaining only time as a fixed effect.
Diffusion of Procedure Innovation Among Cardiac Catheterization Operators at the Veterans Health Administration · University of Washington · PhD
Considered and rejected
Considered and rejected: Rejected mixed-effects logistic regression stratified across all wealth quintiles because the model could not converge at all quintile strata
ASSESSING THE PROVISION AND EQUITABILITY OF PRIMARY CARE IN A LOW-RESOURCE SETTING · Johns Hopkins · PhD
Sparse cluster sizes and low cluster counts undermine higher-level estimation
Analyses suffered when applied to datasets with too few clusters or low subject density per group, such as singletons or under thirty classrooms. These data limitations caused computational instability during Markov chain Monte Carlo sampling and left cross-level interactions severely underpowered.
Considered and rejected
Considered and rejected: Rejected multilevel (hierarchical) modeling due to overlapping participation across ECA categories and small subgroup sizes, opting for single-level multivariate OLS regression.
Exploring the Impact of Extracurricular Activities on Academic Outcomes within the Unique Context of an Independent School · Texas Tech · PhD
Considered and rejected
Considered and rejected: Multilevel modeling with dyad members nested within the dyad was considered and attempted, but decided against because models failed to converge due to insufficient statistical power for 3-way cross-level interactions.
Examining Mother-Adolescent Emotion Dynamics in the Context of Perturbations at Two Time Scales: Parent-Adolescent Conflict and Major Life Stress · Queen's University · PhD
Considered and rejected
Considered and rejected: Rejected using multilevel / hierarchical linear modeling due to small sample size and insufficient number of classroom clusters (<20-30 clusters).
Effectiveness and responsiveness of a fraction vocabulary intervention for students experiencing mathematics difficulty in grade 4 · UT Austin · PhD
Tried and failed
multilevel multinomial logistic regression with random effects applied to small sample clustered longitudinal data. Outcome: unstable. Reason: cluster-period random effects caused computational instability and MCMC sampling failures in small samples
Causal inference methods for cluster-randomized trials under complex selection mechanisms · Penn · PhD
Considered and rejected
Considered and rejected: Rejected Hierarchical Mixed-Effects (Multi-Level) Models for empirical data because singleton patterns at the industry-country level confound random intercepts with overall error terms.
The economics of industrial cyberespionage · Oxford · PhD
Violations of distributional assumptions and predictor collinearity cause estimation bias
Multilevel specifications were rejected when residuals violated assumptions of independence, homoscedasticity, and normality, or when latent variables showed poor fit indices. Severe multicollinearity among predictor scales and estimation bias from lagged dependent variables also led researchers to abandon the approach.
Considered and rejected
Considered and rejected: Rejected a three-block hierarchical regression including state FIPS, Title IV status, programmatic completion rates, and student-to-faculty ratios due to severe missingness and multicollinearity/collinearity.
Factors That Influence Retention Outcomes Among Black Transfer Students at Predominantly White Institutions Using the Culturally Engaging Campus Environments (CECE) Model · Virginia Tech · Doctorate
Tried and failed
multilevel structural equation modeling applied to longitudinal survey diary data. Reason: Poor model fit indices for latent variables
Examining the Benefits of Cisgender Allyship Behavior for Allies of Transgender and Gender Non-conforming Individuals · Texas Tech · PhD
Tried and failed
multilevel modeling with collinear scale covariates applied to dyadic longitudinal survey data. Outcome: unstable. Reason: simultaneous inclusion of highly correlated coping and relationship quality scales caused severe multicollinearity
It takes two : how gender, daily stress processes, and dyadic coping shape well-being in same-sex and different-sex marriages · UT Austin · PhD
Considered and rejected
Considered and rejected: Decided against classical hierarchical linear models (multilevel models) because requirements for independence, homoscedasticity, and normality of residuals were violated, risking model instability
Considered and rejected
Considered and rejected: Rejected compound symmetric covariance structure in linear mixed-effects modeling in favor of a first-order autoregressive structure due to spatial correlation decay between distant disc levels.
Development of Imaging-Based Models for Analyzing the Spatiotemporal Function of Intervertebral Discs · Duke · PhD
Random slope specifications exceed data support and cause singular fits
Specifying random slopes for time or predictors frequently caused non-convergence and singular fits because datasets lacked sufficient within-person observations. Including multiple random slopes alongside random intercepts overparameterized the models and failed to improve overall fit.
Tried and failed
maximal random slopes mixed-effects linear regression applied to hierarchical experimental survey response data. Outcome: did not converge. Reason: model complexity exceeded data support causing singular or non-converging optimization
Spanish in the college classroom: Investigating the linguistic ideologies behind students’ perceptions of language instructors · Texas Tech · PhD
Considered and rejected
Considered and rejected: Modeling random slopes in multilevel regression models, rejected due to model convergence issues caused by having only two within-person observations.
Comparing the Utility of Scoring Methods for the Hinting Task in a Heterogeneous Clinical Sample · Virginia Tech · Master's
Considered and rejected
Considered and rejected: Rejected multilevel model with both random slopes and random intercepts because it included too many random effects for dataset size
Taking Flight: Overcoming Challenges in Airport Development · Harvard · PhD
Mixed-effects specifications underperform simpler fixed-effects models
In multiple evaluations, mixed-effects models showed signs of overfitting, lower classification precision, and worse parameter error than direct fixed-effects alternatives. Researchers also noted concerns about computational burden and inconsistency if unobserved cluster heterogeneity correlated with covariates.
Lost to a baseline
Fixed-effects parameter estimates derived from mixed-effects models yielded worse MAB and RMSE than models fit directly with fixed effects alone
Modeling Stem Taper of Southern Appalachian Red Spruce · Virginia Tech · Master's
Considered and rejected
Considered and rejected: Decided against mixed-effects GLMs (with sampling round and site as random effects) in favor of fixed-effects GLMs due to observed indicators of model overfitting.
Advancing Rural Public Health: From Drinking Water Quality and Health Outcome Meta-analyses to Wastewater-based Pathogen Monitoring · Virginia Tech · Doctorate
Considered and rejected
Considered and rejected: Rejected mixed-effects models for longitudinal whole-brain lesion analysis due to high computational complexity and potential inconsistency if unobserved cluster heterogeneity correlates with covariates.
Advances in statistical methods for large-scale binary-valued neuroimaging data · Oxford · PhD
Participant migration across clusters violates static hierarchical nesting
Longitudinal student mobility between schools violated the rigid single-campus grouping assumptions required by standard multilevel models. As participants changed locations across study waves, baseline school characteristics ceased being representative and thousands of mobile students had to be excluded.
Lost to a baseline
12,572 students (out of 128,611) were excluded from the longitudinal high school cohort due to moving school campuses during the 4-year study window, as the multilevel model required a single campus grouping factor.
Examining equity of access and participation in computer science education · UT Austin · PhD
Considered and rejected
Considered and rejected: Temporally fixed multilevel model for regression analyses was rejected because base-year school-level characteristics ceased being representative as students migrated between high schools across waves.
Spatial Preference And Spatial Choice: Class-Based Differences In How U.s. High School Students Choose College · Penn · PhD
Left open by the authors
Problems the authors named and did not get to.
Left open
Conduct polynomial or semiparametric regression to test for curvilinear too-much-of-a-good-thing effects of work engagement on organizational outcomes. Blocker: Requires the private multilevel organizational survey dataset used in the thesis.
High-quality leader-member exchange relationship as a key to employee work engagement · University of Vaasa · PhD
Left open
Apply multilevel modeling to the high school survey dataset to examine higher-level teacher effects on math achievement goals and attitudes. Blocker: Requires access to the private high school student and teacher survey dataset.
An examination of high school students’ achievement goal profiles and how they relate to their attitudes toward mathematics · UT Austin · PhD
Left open
Apply multilevel modeling with bootstrapping to analyze within-group variance and non-independence in the cocaine treatment group study dataset. Blocker: Requires private clinical trial dataset collected for the thesis
The role of group cohesion and group therapeutic alliance in a transtheoretical model group treatment for cocaine use · UT Austin · PhD
Left open
Establish theoretical conditions and optimal weighting sequences for Multilevel Monte Carlo when beta <= 1 in pharmacokinetic experimental design models. Blocker: None
Variational, Monte Carlo and policy-based approaches to Bayesian experimental design · Oxford · PhD
Left open
Evaluate mixed-effects conformal inference and BNN frameworks on diverse ML algorithms and multi-level hierarchical structures using public data. Blocker: None
Prediction and monitoring with machine learning under complex dependencies for swine production data · Iowa State · PhD
Left open
Extend the ECov hierarchical regression methodology to other generalized linear model families beyond logistic and Gaussian regression. Blocker: None
Bayesian Linear Modeling in High Dimensions: Advances in Hierarchical Modeling, Inference, and Evaluation · MIT · PhD
Left open
Compare mixed-effects location-scale models to CRVE methods to evaluate if CCREMs can relax exogeneity and homoscedasticity simultaneously via covariate centering. Blocker: None
A comparison of methods for centering covariates in cross-classified random effects models · UT Austin · PhD
Left open
Analyze fMRI mixed-effects model variance components against physical variances and evaluate residual patterns after mixed-effects partitioning. Blocker: None
Extracting Feature Vectors From Event-Related fMRI Data to Enable Machine Learning Analysis · Virginia Tech · Doctorate
Who has worked on this
Authors of the public theses cited on this page, most-cited first.
- Andres Inzunza · Harvard · Master's · 2026
PERIMETRIC OUTCOMES OF MELBOURNE RAPID FIELDS IN PATIENTS WITH GLAUCOMA and DEMOGRAPHIC AND DIAGNOSTIC CHARACTERISTICS OF VIRTUAL REALITY VISUAL FIELD TESTING IN PRIMARY CARE GLAUCOMA SCREENING - Daniel Kane Forman · Texas Tech · PhD · 2026
Exploring the Impact of Extracurricular Activities on Academic Outcomes within the Unique Context of an Independent School - Alexandra B. Tighe · Queen's University · PhD · 2026
Examining Mother-Adolescent Emotion Dynamics in the Context of Perturbations at Two Time Scales: Parent-Adolescent Conflict and Major Life Stress - Dane Isenberg · Penn · PhD · 2026
Causal inference methods for cluster-randomized trials under complex selection mechanisms - Kimberly Dawn Clark · Virginia Tech · Doctorate · 2026
Factors That Influence Retention Outcomes Among Black Transfer Students at Predominantly White Institutions Using the Culturally Engaging Campus Environments (CECE) Model - Rory MacAndrew McKemey · Virginia Tech · Master's · 2026
Comparing the Utility of Scoring Methods for the Hinting Task in a Heterogeneous Clinical Sample - Jussi Tanskanen · University of Vaasa · PhD · 2026
High-quality leader-member exchange relationship as a key to employee work engagement - Ashlinn Seree Peters · Texas Tech · Master's · 2025
Likes, Looks, and Learning: Associations of Adolescent Engagement in Disordered Eating with Parent, Sibling, Peer, and Social Media Influences - Mike Sullivan-Royston · Carleton University · Master's · 2025
Personality-Goal Fit: Benefits and Consequences of Aligning Personality Traits and Personal Goals - Taylor Danae Clark · Texas Tech · PhD · 2025
Spanish in the college classroom: Investigating the linguistic ideologies behind students’ perceptions of language instructors - Danyang Zhang · Iowa State · PhD · 2025
Prediction and monitoring with machine learning under complex dependencies for swine production data - Rola Yaman · Queen's University · Master's · 2024
Immigrants’ Use of Online Mental Health Services during the COVID-19 Pandemic
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.