Library · Machine learning & AI
What Variational Inference got wrong, from 53 theses
Summary written by a model from the records below: The records document multiple recurring limitations when applying variational inference across probabilistic modeling, deep generative architectures, and statistical estimation. Major difficulties include posterior variance underestimation from mean-field assumptions, optimization instability in stochastic algorithms, posterior collapse in autoencoders, and inferior estimation accuracy relative to sampling baselines. These records come from graduate theses, 2021 to 2026, each labeled with its degree where the repository states it, and each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Plus 8 problems the authors left open, listed at the end.
Failure modes, by theses reporting them
Select a mode to jump to its records.
Mean-field factorizations underestimate posterior variance and fail to capture parameter correlations
Assuming independent parameters in mean-field variational inference consistently leads to artificially narrow credible intervals and severe undercoverage of uncertainty. When latent variables or features exhibit strong correlations, factorized approximations fail to capture joint dependencies and underperform joint or structured variational models.
Tried and failed
stochastic variational inference for point processes applied to hierarchical temporal point process models. Reason: systematically underestimates posterior variance leading to poor credible interval coverage
Hawkes Processes in Marketing · Penn · PhD
Tried and failed
mean-field stochastic variational inference applied to Bayesian models with correlated parameters. Reason: Factorized posterior distributions inherently underestimate uncertainty when latent parameters exhibit strong correlations
Deciphering the nature of cell state transitions in single cells using quantitative modeling of temporal dynamics · EPFL · PhD
Tried and failed
mean-field variational inference with KL divergence applied to spatial survival models. Reason: Severely underestimated posterior variance and exhibited residual deviations at distribution tails.
Advancements on the Interface of Computer Experiments and Survival Analysis · Virginia Tech · Doctorate
Tried and failed
mean-field stochastic variational inference applied to joint manifold and velocity estimation. Reason: mean-field approximation failed to capture strong posterior correlations between latent parameters, underestimating uncertainty
From Phase to Fate: Multi-Modal Characterisation of Cell Cycle Dynamics and Their Role in Early Development · EPFL · PhD
Tried and failed
mean-field variational inference applied to correlated sparse linear regression. Reason: mean-field assumption underestimates posterior variance under feature correlation, causing severe empirical undercoverage
Theoretical and methodological advances in Bayesian semiparametrics with variational inference · Imperial · PhD
Tried and failed
mean-field variational inference for neural ODEs applied to uncertainty quantification in dynamic systems. Outcome: did not generalise. Reason: mean-field approximation failed to capture posterior correlations needed for accurate forecasting compared to MCMC
A new way to do epidemic modeling · MIT · PhD
Lost to a baseline
Mean field variational Bayes underperformed the joint Kronecker variational model under misspecified low-rank perturbations, requiring 5946 iterations for r=1 versus 303 for the joint model.
Geometric Methods in MCMC and Variational Bayes for Multiway Data · Cornell · PhD
Lost to a baseline
SVB marginal credible interval coverage for non-zero coefficients (0.770) was lower than MCMC (0.928) on survival analysis setting 1 with c=0.25 due to variational variance underestimation.
Variational bayes for high-dimensional linear models · Imperial · PhD
Considered and rejected
Considered and rejected: Variational Bayes ('meanfield') was rejected for final epidemic parameter inference due to ignoring posterior correlations and underestimating credible intervals relative to HMC.
MCMC methods: graph samplers, invariance tests and epidemic models · Imperial · PhD
Tried and failed
mean-field variational inference with diagonal covariance applied to spatial transformation parameter estimation. Outcome: worse than baseline. Reason: diagonal covariance variational posterior was too restrictive to capture dependencies for accurate registration
Bayesian and deep generative modelling for image registration, with a focus on uncertainty quantification & similarity learning · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected mean-field variational inference (SVI) for velocity-learning because it yielded artificially narrow credible intervals and failed to capture joint parameter correlations between degradation rate and angular speed.
Deciphering the nature of cell state transitions in single cells using quantitative modeling of temporal dynamics · EPFL · PhD
Considered and rejected
Considered and rejected: Rejected mean-field variational posterior factorization in ADVI for ChronoStrain in favor of a fully joint Gaussian approximation to preserve time-series coherence
Algorithms for Reconstructing Biological History from Genomic Data · MIT · PhD
Tried and failed
mean-field variational Bayes applied to PDE parameter estimation. Reason: Severely underestimated posterior variance and uncertainty due to assuming complete independence between parameters
Interpretable models for spatially dependent and heterogeneous phenomena · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected mean-field SVI variational family for velocity learning because it assumes parameter independence, leading to overconfident posterior uncertainty bounds.
From Phase to Fate: Multi-Modal Characterisation of Cell Cycle Dynamics and Their Role in Early Development · EPFL · PhD
Considered and rejected
Considered and rejected: rejected standard mean-field variational inference in deep generative OSBM in favor of structured stochastic variational inference (SSVI)
Towards Efficient Continual Learning in Deep Neural Networks · Duke · PhD
Stochastic and particle variational optimization suffers from instability and divergence
High parameter dimensionality, non-convex loss landscapes, and stochastic sampling noise cause variational optimization to diverge or become trapped in poor local minima. Particle-based approaches and normalizing flows also encounter mode collapse, excessive computation times, or unstable feedback loops without delicate tuning.
Tried and failed
variational mutual information estimators applied to multimodal cross-modal representation learning. Outcome: unstable. Reason: estimators underestimate due to batch size limits or diverge in high mutual information regimes
Multimodal Representation Learning for Medical Image Analysis · MIT · PhD
Tried and failed
non-amortized stochastic variational inference applied to latent variable sequence modeling. Outcome: worse than baseline. Reason: insufficient local parameter updates during optimization compared to amortized inference networks
Structure Modeling for Language Models · Harvard · PhD
Tried and failed
joint end-to-end backpropagation through amortized variational inference applied to variational dropout in deep neural networks. Outcome: unstable. Reason: gradients from the inference network backpropagating into the generative decoder adversely affected training stability
Deep neural networks with contextual probabilistic units · UT Austin · PhD
Tried and failed
deep convolutional variational autoencoder applied to multivariate time series feature extraction. Outcome: did not converge. Reason: gradient backpropagation degradation in overly deep architectures led to significantly higher validation loss
Investigating the Brain States Behind FMRI Temporal Dynamics Using Frame-based Analysis Methods and Variational Autoencoders · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Standard Variational AutoEncoder (VAE) with KL-divergence regularization on a single latent variable was rejected for inverse continuous localization because stochastic sampling caused noisy loss landscapes.
Condition monitoring for dry cask storage using helical guided ultrsonic waves · UT Austin · PhD
Considered and rejected
Considered and rejected: Rejected standard variational mutual information approximation for attention control because estimating posterior p(lt|st) without ground-truth state created unstable feedback loops and mode collapse.
Task Generalized MDPs for Multi-Task Reinforcement Learning · Georgia Tech · PhD
Tried and failed
annealing smoothing variance during variational inference applied to latent space inverse problem solving. Reason: sample quality failed to improve at very small smoothing variance values
Applications of deep generative models : inverse problems, compression, and beyond · UT Austin · PhD
Tried and failed
stochastic variational inference with normalizing flows applied to sequential surrogate-based parameter estimation. Outcome: unstable. Reason: lack of temperature annealing caused mode collapse, overconfident posteriors, and optimization divergence
Neural network based surrogates for scalable Bayesian inference on a complex malaria model · Imperial · PhD
Tried and failed
stochastic variational inference applied to high-dimensional parameter estimation. Outcome: unstable. Reason: optimization instability increases significantly as the number of simultaneous input parameters scales up
Tried and failed
variational free energy for hidden Markov model selection applied to time-series state inference. Reason: models with equivalent free energy yielded divergent posterior state dynamics and inconsistent downstream interpretations
Multiscale neural dynamics and task-dependent states in human MEG via time-delay embedding · EPFL · PhD
Tried and failed
variational inference with mean-field Gaussian prior applied to full-waveform inversion with poor initialisation. Outcome: did not converge. Reason: Severe non-convexity and local minima when starting from a flat homogeneous prior
Variational tomography; a low-cost approach to estimate reconstruction uncertainty with an application to full-waveform inversion · Imperial · Doctorate
Continuous latent spaces and prior regularization distort trajectories and physical representations
Enforcing variational priors and continuous latent spaces often warps dynamical trajectories, creates discontinuities across time steps, or produces blurry generation outputs. These restrictive variational representations frequently degrade downstream prediction and structural analysis compared to standard autoencoders or simpler deterministic baselines.
Tried and failed
continuous conditional variational autoencoder applied to recursive trajectory and motion generation. Outcome: did not generalise. Reason: Continuous latent space caused blurry generations and failed to follow conditioning velocity commands over recursive rollouts
Generative Latent Motion Planning and Reinforcement Learning for Legged Locomotion · MIT · PhD
Tried and failed
variational auto-encoders for counterfactual generation applied to anomaly repair and explanation. Outcome: worse than baseline. Reason: struggled to generate high-quality repairs compared to diffusion models
Reliable Anomaly Detection with Explanation and Feedback · Penn · PhD
Tried and failed
variational autoencoder for latent trajectory modeling applied to cell shape dynamics over time. Outcome: worse than baseline. Reason: prior regularisation warped latent dynamical trajectories compared to standard autoencoders
Computing Interpretable Representations of Cell Morphodynamics · Imperial · PhD
Tried and failed
variational autoencoders for continuous dynamical systems applied to phase space trajectory modeling. Reason: failed to capture global topology and caused discontinuities across consecutive time steps
Conservation laws as inductive biases · Imperial · PhD
Tried and failed
MLP-based variational autoencoder applied to cross-dataset structural anomaly detection. Outcome: did not generalise. Reason: model overfitted to source structure distributions and failed on unseen structural data
Machine learning tools for identifying structural artifacts in data · Imperial · Master's
Lost to a baseline
Variational Autoencoders (VAEs) lose to standard Autoencoders (AEs) when training SNR and testing SNR perfectly match due to gaps in the latent space
Model and data driven approaches to wireless image transmission · Imperial · Doctorate
Considered and rejected
Considered and rejected: Rejected standard Variational Autoencoder (VAE) architecture for turbofan RUL estimation because decoders fail on temporal state projections; replaced decoder with a regressor network (RVE).
Uncertainty quantification of faults in rotating machines · Texas Tech · PhD
Tried and failed
surrogate modeling on variational autoencoder latent space applied to property prediction from spatial representations. Outcome: worse than baseline. Reason: nonlinear latent embedding degraded forward predictions and increased uncertainty compared to linear dimensionality reduction
Neural Inverse Microstructure Design with Bayesian Scale-Bridging · Georgia Tech · PhD
Tried and failed
variational autoencoder on covariance-filtered geometric features applied to single-cell chromatin dispersion clustering. Outcome: worse than baseline. Reason: removing correlated features discarded informative signals needed to separate characteristics beyond simple cluster counts
Categorizing the dispersion of open chromatin usage patterns of Treg related genes using an variational autoencoder based algorithm · Harvard · Master's
Gaussian and unimodal variational families fail to express multimodal or non-continuous posteriors
Symmetric Gaussian and unimodal variational approximations underfit complex models and fail to capture multimodal or asymmetric parameter spaces. Practitioners consequently rejected these variational families because they provide uninformative uncertainty quantification and cannot accommodate discrete reference priors.
Tried and failed
weight-space mean-field variational inference applied to transformer neural networks. Outcome: worse than baseline. Reason: the variational approximation severely underfits across predictive benchmarks
Probabilistic learning and generation in deep sequence models · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected frequentist linear mixed models (e.g., lme4) and variational inference due to intractable or unreliable uncertainty estimation and p-value derivation in high-dimensional hierarchies.
Studying the tissue-specificity of cancer driver genes through KRAS and genetic dependency screens. · Harvard · PhD
Considered and rejected
Considered and rejected: MC dropout variational Bayes for NN-GLS uncertainty quantification was rejected because binary variational approximation of continuous weight posteriors is too crude to yield valid interval estimates.
SPATIAL METHODS WITH GEOPHYSICAL AND GENOMIC APPLICATIONS · Johns Hopkins · PhD
Considered and rejected
Considered and rejected: Point-wise variational inference / inducing point approximations due to oversmoothing and loss of uncertainty quantification
Modernizing Latent Gaussian Process Inference for Non-Gaussian Responses · Virginia Tech · Doctorate
Considered and rejected
Considered and rejected: Rejected Variational Bayesian Inference (VBI) and Laplace approximation due to reliance on unimodal/Gaussian parametric approximations that underestimate uncertainty in non-linear, multimodal ODE posteriors.
Bayesian Machine Learning for Gender-Stratified Mechanistic Epidemiological Models Using Public Health Data. · Carleton University · Master's
Tried and failed
variational autoencoders for sequential modeling applied to multimodal agent state estimation. Outcome: did not generalise. Reason: failed to capture multimodal distributions under sparse and partial observations
Trajectory Modeling using Generative Approaches for Scheduling, Planning, and Multi-Agent Systems · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Variational inference / simplified posterior distributions for BNN PDE discovery, because it yields less informative derivative uncertainty quantification despite computational savings
Bayesian Machine Learning Algorithms for Uncertainty Quantification, Optimization, and Equation Discoveries in Engineering Physics · Cornell · PhD
Considered and rejected
Considered and rejected: Decided against Variational Inference for the scale stability model because VI approximates posteriors symmetrically with Gaussians, which is inappropriate for psychological space.
Machine Learning for Psychophysical Scaling with Ordinal Comparisons · Tübingen · PhD
Considered and rejected
Considered and rejected: Rejected continuous variational approximations (e.g., Gaussian families) for reference priors because true reference priors are discrete and variational approximations fail to discover diverse models
The Principles of Learning on Multiple Tasks · Penn · PhD
Variational approximations produce inferior posterior fidelity and topic quality compared to Markov Chain Monte Carlo
Variational inference often achieves worse negative log-likelihood, lower topic coherence, and higher topic overlap than Markov Chain Monte Carlo and Gibbs sampling. The method can fail to converge or risk propagating bias, leading researchers to favor sampling algorithms for reliable estimation.
Tried and failed
neural topic modeling using variational autoencoders applied to long academic documents. Outcome: worse than baseline. Reason: consistently produced negative normalized pointwise mutual information and low topic coherence scores
Topic Modeling for Heterogeneous Digital Libraries: Tailored Approaches Using Large Language Models · Virginia Tech · Master's
Tried and failed
variational Bayes LDA topic modelling applied to sparse multi-decade longitudinal text corpora. Outcome: unstable. Reason: computationally unstable and unrepeatable compared to Gibbs sampling on sparse corpora
The application of quantitative methods to the study of firm-level adaptation and investment preferences in the energy sector · Imperial · PhD
Tried and failed
variational inference with laplace approximation applied to bayesian hierarchical spatiotemporal models. Outcome: did not converge. Reason: failed to converge to sensible parameter values without heavy customisation, requiring MCMC instead
Spatiotemporal modelling of all-cause and cause-specific mortality in England · Imperial · PhD
Lost to a baseline
Markov Chain Monte Carlo (MCMC) yielded models that better fit observed well data compared to variational inference stochastic gradient optimiser (Wingate et al., 2016).
Simulation of geothermal reservoirs with data assimilation and reduced order modelling · Imperial · PhD
Lost to a baseline
On the Titan GPU dataset, KL divergence variational inference achieved an NLL of 2411.86 and took 96.05 minutes, performing worse than standard HMC which achieved an NLL of 2040.68 in 20.89 minutes.
Advancements on the Interface of Computer Experiments and Survival Analysis · Virginia Tech · Doctorate
Considered and rejected
Considered and rejected: Variational inference / expectation propagation for fitting Bayesian state-space models, rejected due to unsuccessful convergence/fitting compared to MCMC (NUTS)
Towards a data-driven personalised management of Atopic Dermatitis severity · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected Gensim Variational Bayes LDA implementation in favor of Mallet Gibbs sampling due to higher topic overlap and lower coherence.
The Satisfaction Levels of Services Provided by Transportation Network Companies (TNCs): A Semantic and Spatial Examination Based on Social Media Data · University at Buffalo · PhD
Variational autoencoders experience severe posterior collapse and destructive over-regularization
Standard priors and heavy auxiliary regularization frequently penalize generative reconstruction objectives and collapse latent representations into shortcuts or single mixture components. These collapse dynamics result in blurry outputs, uninformative latent codes, and degraded sample generation across visual and textual domains.
Tried and failed
autoregressive latent covariance structure applied to variational autoencoders. Reason: did not prevent posterior collapse when it occurred in standard VAE baseline
Deep Time: Deep Learning Extensions to Time Series Factor Analysis with Applications to Uncertainty Quantification in Economic and Financial Modeling · Virginia Tech · Master's
Tried and failed
information bottleneck and variational representation learning applied to out-of-distribution visual policy learning. Outcome: did not generalise. Reason: Representations collapsed into action-only shortcuts or encoded high-variance visual distractors.
From Pixels to Partners: A Hierarchical Approach to Adaptive Human-Robot Collaboration · Virginia Tech · Doctorate
Tried and failed
variational autoencoder with decoder word dropout applied to text generation and reconstruction. Outcome: no signal. Reason: severe posterior collapse leading to poor reconstruction quality despite high dropout rates
Controlling Neural Language Generation · MIT · PhD
Tried and failed
Mixture-of-Gaussians approximate posterior with Dirichlet weights applied to variational autoencoder latent space. Reason: Optimization collapsed mixture weights to a single component, failing to improve posterior expressiveness
Human-controllable and structured deep generative models · Imperial · PhD
Tried and failed
high auxiliary regularization weight in variational autoencoder applied to image generation and representation learning. Outcome: worse than baseline. Reason: overly strong regularization heavily penalized the primary reconstruction objective, degrading sample generation quality
Capsule Networks: Framework and Application to Disentanglement for Generative Models · Virginia Tech · Master's
Left open by the authors
Problems the authors named and did not get to.
Left open
Extend LDML estimation and inference for estimand-dependent nuisances to non-smooth settings with multi-valued parameter regimes. Blocker: Lacks specific theoretical approach or algorithmic formulation for handling non-smooth multi-valued parameter regimes
Machine Learning Methods for Data-driven Decision Making: Contextual Optimization, Causal Inference, and Algorithmic Fairness · Cornell · PhD
Left open
Extend nested sampling Bayesian inference to simultaneously model and fit multiple cognitive tasks across experimental paradigms. Blocker: The unfinished work lacks specific target cognitive tasks, shared parameter structures, or concrete datasets to implement.
Left open
Develop and compare methods such as variational MAP estimation, Bayes factors, or reversible jump MCMC to summarize binary indicator matrix MCMC posterior samples. Blocker: None
Probabilistic approaches to multi-study genomic matrix decompositions and single-cell RNA-sequencing analysis · Harvard · PhD
Left open
Extend supervised LDA to time-varying parameters using dynamic modeling or rolling windows and compare variational inference with MCMC/Gibbs sampling. Blocker: None
ESSAYS IN FINANCIAL ECONOMETRICS · Penn · PhD
Left open
Develop statistical inference frameworks to estimate cluster variance and confidence intervals for UMAP embedding outputs. Blocker: Lacks a concrete theoretical framework or formulation to guide the development of statistical inference on model-free embeddings
Ensemble Methods for Latent Structure Detection from Heterogeneous Genomic and Phenotypic Data · Harvard · PhD
Left open
Explore alternative sampling and normalization methods to optimize the Bayesian Inverse Reinforcement Learning inference process across different interaction settings. Blocker: Vague task specification with no defined alternatives or evaluation targets
Inferring the Human's Objective in Human Robot Interaction · Virginia Tech · Master's
Left open
Implement variational inference for the bespoke Bayesian hierarchical Gaussian mixture model to improve computational scalability over MCMC. Blocker: None
Clustering of cardiometabolic and renal risk factors · Imperial · PhD
Left open
Reduce inference computational cost during structured approximate cross-validation for very large graphical models. Blocker: No specific algorithmic approach or target architecture is defined
Faster and easier: cross-validation and model robustness checks · MIT · PhD
Who has worked on this
Authors of the public theses cited on this page, most-cited first.
- Maxine Leonardi · EPFL · PhD · 2026
From Phase to Fate: Multi-Modal Characterisation of Cell Cycle Dynamics and Their Role in Early Development - Alex Russell Lederer · EPFL · PhD · 2024
Deciphering the nature of cell state transitions in single cells using quantitative modeling of temporal dynamics - Yueyao Wang · Virginia Tech · Doctorate · 2023
Advancements on the Interface of Computer Experiments and Survival Analysis - Fabrice Jean Guibert · EPFL · PhD · 2026
Multiscale neural dynamics and task-dependent states in human MEG via time-delay embedding - Omodolapo David Ajayi · Texas Tech · PhD · 2026
Uncertainty quantification of faults in rotating machines - Andrew Harrison Cooper · Virginia Tech · Doctorate · 2026
Modernizing Latent Gaussian Process Inference for Non-Gaussian Responses - Sakhi Sanjay Mittal · Carleton University · Master's · 2026
Bayesian Machine Learning for Gender-Stratified Mechanistic Epidemiological Models Using Public Health Data. - Sagar Parekh · Virginia Tech · Doctorate · 2026
From Pixels to Partners: A Hierarchical Approach to Adaptive Human-Robot Collaboration - Quinn Simonis · Cornell · PhD · 2025
Geometric Methods in MCMC and Variational Bayes for Multiway Data - Wenlong Chen · Imperial · PhD · 2025
Probabilistic learning and generation in deep sequence models - Rebecca Danning · Harvard · PhD · 2025
Ensemble Methods for Latent Structure Detection from Heterogeneous Genomic and Phenotypic Data - Adam Joseph Miller · MIT · PhD · 2025
Generative Latent Motion Planning and Reinforcement Learning for Legged Locomotion
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.