Library · Machine learning & AI
What Markov Models got wrong, from 53 theses
Summary written by a model from the records below: The records document practical breakdowns and modeling limitations encountered when applying Markov models across various computational and engineering domains. Researchers frequently discarded or modified these approaches due to computational intractability, invalid memoryless assumptions, numerical instability, and sensitivity to data sparsity. These records come from graduate theses, 2021 to 2026, each labeled with its degree where the repository states it, and each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Plus 9 problems the authors left open, listed at the end.
Failure modes, by theses reporting them
Select a mode to jump to its records.
Computational intractability and state space explosion
Scaling Markov models to large state spaces or continuous settings created severe computational bottlenecks that prevented exact evaluation and planning. Authors frequently abandoned full formulations because calculating matrix exponentials, path sums, or policy spaces became computationally infeasible.
Considered and rejected
Considered and rejected: Rejected recursive Hidden Markov Model (HMM) parameter estimation due to high computational expense scaling with domain/model space, replacing it with deep dynamic autoencoders with replay buffers.
Cooperative and Distributed Algorithms for Dynamic Fire Coverage using a Team of UAVs · University of Nevada, Reno · Master's
Lost to a baseline
Markov networks and Markov random fields could not be used due to computational infeasibility with large species numbers and errors with common/rare species, defaulting to simpler GLM and correlation methods.
Responses of ecological communities to deforestation: an empirical and theoretical perspective · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected treating full sequence dynamics as an explicit Markov chain path trajectory because summing over all intermediate sequence paths is computationally intractable.
Unsupervised inference methods for protein sequence data · Politecnico di Torino · PhD
Considered and rejected
Considered and rejected: Rejected naive full state space exploration in allotetraploid HMM due to exponential computational complexity; constrained state space using a first-order Markov chain on covering reads.
Unsupervised learning with high-throughput sequencing data · Iowa State · PhD
Considered and rejected
Considered and rejected: Markov-Chain Monte Carlo (MCMC) sampling for Bayesian harmonic inference; rejected because it became computationally intractable when scaling across spatially coherent networks of thousands of nodes.
Spatiotemporal tidal prediction and analysis through physics-informed machine learning · Oxford · PhD
Considered and rejected
Considered and rejected: Rejected direct parameterization and likelihood evaluation of the CTMC generator L due to intractability, numerical instability of matrix exponentials, and the Markov chain embedding problem.
Three Essays of Bayesian Inference on Dynamical System, Continuous Time Markov Chain, and Low Dimensional Structure · Duke · PhD
Considered and rejected
Considered and rejected: Rejected standard Markov Processes because they require constant transition rates (exponential distributions) and suffer from state-space explosion.
Risk modelling of safety critical systems for life extension in offshore oil and gas · Nottingham
Considered and rejected
Considered and rejected: Rejected Markov models because of constant-rate memoryless assumptions and state space explosion when modeling complex interacting asset sub-processes
Renewals scheduling for high-speed railway assets · Nottingham · Doctorate
Considered and rejected
Considered and rejected: Rejected computing VaR exactly via the worst possible history in the continuous state space Markov chain, choosing Monte Carlo estimation instead due to computational intractability.
Risk-sensitive and robust model-based reinforcement learning and planning · Oxford · PhD
Tried and failed
sequential convex programming and off-the-shelf QCQP solvers applied to large parametric Markov decision processes. Outcome: did not converge. Reason: conservative step sizing and numerical instability caused timeouts on large models
Convex optimization meets formal methods : verification, synthesis, and learning in Markov decision processes · UT Austin · PhD
Considered and rejected
Considered and rejected: Rejected full Markov Decision Process (MDP) formulation for contested logistics because solving it is currently computationally intractable or requires oversimplification.
GAME-THEORETIC MODELS FOR RAPID OPERATIONAL AIRLIFT NETWORK DESIGN IN CONTESTED ENVIRONMENTS · Naval Postgraduate School · Master's
Considered and rejected
Considered and rejected: Rejected standard discrete probabilistic Markov Decision Processes (MDPs) for multi-queue network control due to the curse of dimensionality and inability to handle non-exponential service distributions or propagation time-delays.
Modeling and Control of Queuing Networks: Applications to Airport Surface Operations · MIT · PhD
Considered and rejected
Considered and rejected: Rejected using Markov Decision Process (MDP) for dynamic pricing due to exponential state space explosion with network size (|V|, |A|) and lead times (L), choosing a fluid approximation model instead.
Marketplace Design for Crowdsourced Same-Day Delivery · Georgia Tech · PhD
Transition rate misspecification and unobserved heterogeneity
Prespecified transition models failed to represent latent heterogeneity, non-stationary reliability shocks, and complex multi-site dependencies. These structural misspecifications caused severe estimation errors and led models to underperform compared to flexible alternatives.
Tried and failed
training separate time-period-specific hidden Markov models applied to temporal sequence prediction. Outcome: worse than baseline. Reason: a single unified model performs as well as or better than separate time-sliced models
Disaggregated short-term travel location prediction · Imperial · PhD
Tried and failed
non-homogeneous hidden semi-Markov models applied to early-stage component degradation tracking. Outcome: worse than baseline. Reason: statistical assumptions failed to capture ground truth wear dynamics during early operational life
Digital Twin-Driven Condition Monitoring Approach for Aircraft Carbon Brakes · Georgia Tech · PhD
Tried and failed
multitype branching process approximation of Markov chains applied to periodic extinction probability boundary estimation. Outcome: did not generalise. Reason: fails to predict dominance boundaries when periodic parameters across compartments have phase shifts or asynchrony
Infectious Disease Dynamics in Heterogeneous Populations: Stochastic, Seasonal, and Structured Epidemic Models with Intervention Strategies · Texas Tech · PhD
Tried and failed
discrete Hidden Markov Model sequential state prediction applied to human compliance dynamics under system errors. Outcome: did not generalise. Reason: model fails to capture sudden non-stationary state shifts caused by negative reliability shocks
Human-Robot Trust in Time-Sensitive Scenarios · Georgia Tech · PhD
Tried and failed
mean-field Markov chain approximations applied to spatial lattice excitation dynamics. Outcome: did not generalise. Reason: mean-field approximations omit critical multi-site spatial defect structures that drive persistent failure states
Tried and failed
standard Markov models of discrete character evolution applied to complex discrete trait evolutionary transitions. Outcome: worse than baseline. Reason: substantially worse fit than hidden-state models due to unobserved rate heterogeneity across lineages
When the Map Fails the Territory: Hidden State Models, Complex Traits and the Evolution of Bird Migration · Virginia Tech · Doctorate
Tried and failed
constrained continuous-time Markov transition model applied to discrete morphological trait evolution. Outcome: worse than baseline. Reason: constrained directional transition rate assumptions fit empirical evolutionary data substantially worse than standard symmetrical rates
A Morphological and Phylogenetic Comparative approach to investigating Branching Architecture Evolution in Monocots A Dissertation · Cornell · PhD
Tried and failed
Bayesian ideal observer hidden Markov model applied to probabilistic bandit decision making. Outcome: worse than baseline. Reason: Severely overestimated policy switching probability compared to empirical behavior
Relationship of striatal dopamine to reward history and action during probabilistic decision making · Harvard · PhD
Tried and failed
hidden Markov model for anomaly detection applied to combustion instability detection. Outcome: did not generalise. Reason: excessive false positive rate classifying normal conditions as anomalies
Trustworthy deep learning for cyber-physical systems · Iowa State · PhD
Considered and rejected
Considered and rejected: Rejected Hidden Markov Models (HMM) for estimating unobserved HCV infection and reinfection risk trajectories because HMMs require a priori specification of transition matrices and functional relationships with covariates.
Implications of reinfection for Hepatitis C elimination among people who inject drugs · Johns Hopkins · PhD
Optimization failures and objective degeneracies in Markov decision processes
Formulating sequential decisions as Markov decision processes often produced degenerate policies, local optima, or unbounded delay behaviors. Algorithms struggled with high policy variance, solver timeouts, and invalid stationary distribution constraints under partial observability.
Tried and failed
Marginal polytope constraints for joint distribution validity applied to partially observed stationary Markov decision processes. Reason: Marginal state-action-state constraints fail to guarantee existence of a valid underlying stationary joint transition distribution.
Towards Credible and Effective Data-Driven Decision-Making · Cornell · PhD
Tried and failed
unconstrained conditional maximum likelihood estimation applied to linear Markov decision process representation learning. Reason: intractable marginal regularity constraints requiring integrals over continuous state spaces to sum to one
Towards provable and practical reinforcement learning with representation learning · UT Austin · PhD
Tried and failed
Markov decision process linear programming formulation applied to individual asset maintenance scheduling. Reason: optimises aggregate state proportions rather than assigning specific actions to discrete individual entities
Pavement maintenance and rehabilitation programming and budget allocation from strategic perspectives : multiobjective optimization models · UT Austin · PhD
Tried and failed
applying nominal policies to uncertain environments applied to partially observable Markov decision processes. Outcome: did not generalise. Reason: nominal control policies lack robustness to parametric model uncertainties, drastically increasing failure probability
Convex optimization meets formal methods : verification, synthesis, and learning in Markov decision processes · UT Austin · PhD
Tried and failed
deterministic Markovian policy modifications applied to constrained correlated equilibrium search. Outcome: worse than baseline. Reason: yields a strictly weaker equilibrium concept than allowing stochastic modifications in constrained games
Game-Theoretic Decision Making in Multi-Agent Systems under Constraints and Welfare Objectives · EPFL · PhD
Tried and failed
unbiased loss estimation in policy search applied to adversarial episodic markov decision processes. Outcome: unstable. Reason: high variance causes random regret to be linear with nonzero probability
Efficient, reliable, and interpretable decision-making for human-autonomy co-existence · UT Austin · PhD
Tried and failed
sequential convex programming with trust regions applied to parametric Markov decision process synthesis. Reason: algorithm converged to a local optimum exceeding the required threshold specification
Convex optimization meets formal methods : verification, synthesis, and learning in Markov decision processes · UT Austin · PhD
Tried and failed
infinite-horizon augmentation without padding applied to dialogue Markov decision processes. Reason: policy values depend on variable episode turn lengths
Reducing Human Labor Cost in Deep Learning for Natural Language Processing · Georgia Tech · PhD
Tried and failed
marginalized importance sampling with nonparametric estimators applied to off-policy evaluation in Markov decision processes. Reason: the estimator does not represent a valid influence function, failing to achieve the semiparametric efficiency bound
STATISTICALLY EFFICIENT REINFORCEMENT LEARNING · Cornell · PhD
Tried and failed
Discounted cost minimization with reachability constraints applied to Markov decision processes. Reason: Agents delay target reaching indefinitely to drive discounted costs to zero, so optimal policies do not exist.
Algorithms for cooperative and competitive autonomous systems · UT Austin · PhD
Violations of the memoryless assumption and duration dependence
Standard first order Markov assumptions proved inadequate for systems where future states depend on historical context, sojourn times, or non-exponential wear. Authors rejected pure Markov processes in favor of semi-Markov frameworks or other history-dependent representations.
Tried and failed
recursive discrete Bayes filters for sequence recognition applied to multimodal verbal and gesture command recognition. Outcome: worse than baseline. Reason: Markov assumption failed for temporal sequences and compounding multi-step pipeline errors degraded system-level accuracy
Multimodal spatio-semantic perception and team performance evaluation for hierarchical human-robot interaction in dynamic environments · UT Austin · PhD
Considered and rejected
Considered and rejected: Rejected Markov model for baseline hazard in favor of semi-Markov conditional hazard h03(t2 - t1) to capture sojourn time dependence.
Evaluation of multivariate longitudinal data accounting for missingness: methods and applications · Boston University · PhD
Tried and failed
Markov chain model applied to infrastructure degradation state prediction. Outcome: worse than baseline. Reason: Inability to capture complex nonlinear dependencies and feature interactions compared to supervised machine learning models.
Application of Data Analytics and Machine Learning Methods to Enhance Decision-Making in Right-Of-Way Acquisition Process and Transportation Asset Management · Georgia Tech · PhD
Tried and failed
low-order discrete Markov chain applied to weather dynamics simulation. Reason: state space was too simple to capture complex temporal fluctuations
Large scale data analytics for resilience of energy networks · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Rejected simulating bad data injection attack propagation using only pure Markov chains because serial node chains cannot properly express parallel tasks and cannot accommodate vastly different time resolutions between preparation (months/years) and execution (hours).
CYBER THREAT PROPAGATION MODELING IN CYBER PHYSICAL SYSTEMS · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Rejected Markov chains / continuous-time Markov models because the memoryless exponential distribution cannot represent Weibull-distributed track degradation or multi-action maintenance history.
Railway track asset management modelling · Nottingham · PhD
Considered and rejected
Considered and rejected: Rejected standard first-order Markov chain models for frailty progression in favor of semi-Markov models because transition hazards depend on sojourn time in the current and previous states.
Frail patients’ hospital resource use, outcomes, and care transitions - An analysis of the English NHS · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected Hidden Markov Models (HMMs) because they overlook long-term dependencies and lack explicit parameter uncertainty modeling at urban scales.
Evaluating the effects of occupancy on energy use and indoor environmental quality in residential building archetypes over spatial and temporal scales · University College Dublin · PhD
Considered and rejected
Considered and rejected: Rejected using duration-dependent transition probabilities in the natural history model because memoryless Markov property limited modeling duration directly, using age-specific proxies instead.
Identifying opportunities to reduce cervical cancer prevention disparities in Western Washington · University of Washington · PhD
Data sparsity, over-parameterization, and feature distortion
Hidden Markov models and decision processes degraded when applied to sparse event sequences, low-frequency observation rates, or excessive state representations. Adding extra features or aggressively compressing feature spaces further undermined state identification and transition accuracy.
Tried and failed
Hidden Markov Model with extra observable features applied to latent behavioural state decoding. Outcome: worse than baseline. Reason: Adding a third observable degraded prediction of active wake states compared to a two-observable model
Investigating behavioural correlates of sleep states in drosophila · Imperial · PhD
Tried and failed
Gaussian Hidden Markov Models with Viterbi decoding applied to hidden state estimation in time-series trajectories. Outcome: data insufficient. Reason: Lack of realistic transition matrices and extremely rare state transitions (<1%) in sequential data
Development of A Trajectory Population Data and its Application in CAV Research · Virginia Tech · Doctorate
Tried and failed
PCA dimensionality reduction on temporal sequence features applied to hidden Markov model state prediction. Outcome: worse than baseline. Reason: Component reduction discarded informative variance and dynamic correlations present in raw sequential features
Development of A Trajectory Population Data and its Application in CAV Research · Virginia Tech · Doctorate
Tried and failed
unstructured hidden Markov models with high state counts applied to continuous movement trait evolution. Outcome: overfit. Reason: over-parameterization caused model rejection and made transition rate interpretation difficult
When the Map Fails the Territory: Hidden State Models, Complex Traits and the Evolution of Bird Migration · Virginia Tech · Doctorate
Considered and rejected
Considered and rejected: Rejected using Hidden Markov Models (HMMs) for behavioral state classification (travel vs stationary) due to insufficiently high GPS fix frequencies (>1 min).
Factors Associated with Space Use and Reproductive Success of American Kestrels in Northern Virginia, USA · Virginia Tech · Master's
Considered and rejected
Considered and rejected: Rejected data-driven Hidden Markov Model (HMM) stride detection due to poor performance on short walking bouts (<30 strides), which make up ~65% of daily strides.
Real world multimodal assessment of physical behavior change in health and disease · EPFL · PhD
Considered and rejected
Considered and rejected: Rejected Hidden Semi-Markov Models (HSMMs) because temporal state transition duration was not critical and HSMMs require significantly more training data.
Development and Comparison of Operator Strategy Models · Duke · PhD
Considered and rejected
Considered and rejected: Full Markov Decision Process (MDP) modeling rejected for contextual bandits with surrogate reward costs due to severe state space constraints, high noise, and sample sparsity causing inaccurate transition models.
Deployable Online Reinforcement Learning Algorithms · Harvard · PhD
Considered and rejected
Considered and rejected: Rejected full Markov Decision Process (MDP) modeling for delayed effects due to limited data and noise, opting instead for engineered bandit rewards.
Deployable Online Reinforcement Learning Algorithms for Use-inspired Research in Digital Health · Harvard · PhD
Sampling non-convergence and poor Markov chain mixing
Markov chain Monte Carlo methods and state transitions suffered from slow mixing, non-ergodic dynamics, and divergent variation sums. Operational disruptions such as dynamic ordering adjustments or chain interruptions introduced sampling biases that prevented convergence to correct target distributions.
Tried and failed
Gibbs sampling applied to linear Gaussian random acceleration systems. Outcome: did not converge. Reason: The Markov chain induced by Gibbs updates is non-ergodic for random acceleration dynamics.
Uncertainty Quantification and Structure Discovery for Scalable Behavior Science · MIT · PhD
Tried and failed
complex Markov chain dynamics applied to causal set growth models. Outcome: did not converge. Reason: divergence of total variation sum prevents well-defined decoherence functional on the full measure algebra
Discrete random spacetimes: covariance and quantization in growth dynamics for causal sets · Imperial · PhD
Considered and rejected
Considered and rejected: Markov Chain Monte Carlo (MCMC) targeting the posterior was rejected for dynamic sequence tracking due to the requirement to resample entire trajectories rather than updating sequentially.
Mapping dynamic brain networks with MEG data using machine learning · Oxford · PhD
Tried and failed
randomizing sparse approximation orderings between MCMC iterations applied to hierarchical Gaussian process inference. Outcome: unstable. Reason: changing orderings between iterations disrupted Markov chain stationarity and prevented burn-in
Deep Gaussian Process Surrogates for Computer Experiments · Virginia Tech · Doctorate
Tried and failed
real-time interruption replica exchange without worker filtering applied to parallel Markov chain Monte Carlo. Reason: sampling actively running chains introduces length bias, converging to erroneous shifted target distributions
Monte Carlo Methods in Practice and Efficiency Enhancements via Parallel Computation · Cambridge · PhD
Considered and rejected
Considered and rejected: Rejected the Flip chain (single-boundary-cell reassignment Markov chain) because random walks mix extremely slowly and consistently produce non-compact districts.
Algorithms for Fair Redistricting · Harvard · PhD
Left open by the authors
Problems the authors named and did not get to.
Left open
Implement and evaluate split-merge algorithms for dynamic state space scaling in Hidden Markov Models for NLP tasks. Blocker: None
Structured State Tracking for Natural Language Understanding · Cornell · PhD
Left open
Extend state-based block Hidden Markov Models with shared transition parameters to more complex reinforcement learning behavioral tasks. Blocker: None
Behavioral Strategies and Neural Mechanisms of Dynamic Foraging · MIT · PhD
Left open
Generalize the distributed estimation heterogeneity framework to handle non-i.i.d. Markov chain data models. Blocker: Theoretical definition and estimators for Markov chain heterogeneity must be developed from scratch without a stated mathematical approach.
Data Heterogeneity in Linear Distributed Estimation and Learning · EPFL · PhD
Left open
Model correlated edge failures across network graphs using Markov chains instead of assuming independent edge failures. Blocker: None
Unveiling Roadway Network Safety: Application of Statistical Physics to Crowdsourced Velocity Data · MIT · PhD
Left open
Scale multivariate Markov chain marginal parametrisation inference to high-dimensional state spaces and large numbers of chains. Blocker: The unfinished work is an open-ended research direction without a concrete proposed algorithm or specification.
Parametrisations for inference on the dependency structures in Multivariate Markov Chains · Nottingham · PhD
Left open
Model high-order and long-term sequential dependencies in Markov chains for recommendation while mitigating state space explosion. Blocker: No specific approach, mathematical formulation, or mitigation technique is defined to handle the state explosion
Semantic Embedded Sequential Recommendation for E-Commerce Products through Mining Customers’ Historical Interactions and Products’ Data · University of Windsor · PhD
Left open
Construct a formal statistical description and Markov partition of turbulence directly from exact coherent structure networks. Blocker: Lacks specific algorithmic methodology for partitioning continuous high-dimensional phase space into symbolic Markov states
On the Generalization of Shadowing to Fluid Turbulence: Practical Methods For Quantifying Dynamical Similarity · Georgia Tech · PhD
Left open
Develop automated structure learning and model selection algorithms for multivariate Markov chain dependency graphs. Blocker: No specific objective function, search strategy, or concrete framework is specified beyond a general goal
Parametrisations for inference on the dependency structures in Multivariate Markov Chains · Nottingham · PhD
Left open
Derive the random-order universal prediction regret and optimal prediction strategy for non-sequential generative models trained on Markov sources. Blocker: None
Universal Prediction in the Age of Large Language Models · EPFL · PhD
Who has worked on this
Authors of the public theses cited on this page, most-cited first.
- Jiale Tao · Nottingham · PhD · 2026
Parametrisations for inference on the dependency structures in Multivariate Markov Chains - Nicholas Jordan Bone · Virginia Tech · Doctorate · 2025
When the Map Fails the Territory: Hidden State Models, Complex Traits and the Evolution of Bird Migration - Md Rauful Islam · Virginia Tech · Doctorate · 2023
Development of A Trajectory Population Data and its Application in CAV Research - Susobhan Ghosh · Harvard · PhD · 2026
Deployable Online Reinforcement Learning Algorithms for Use-inspired Research in Digital Health - Emily Buttriss · Nottingham · Doctorate · 2026
Renewals scheduling for high-speed railway assets - Anna Maria Maddux · EPFL · PhD · 2026
Game-Theoretic Decision Making in Multi-Agent Systems under Constraints and Welfare Objectives - Marco Bondaschi · EPFL · PhD · 2026
Universal Prediction in the Age of Large Language Models - Jamie Tucker-Foltz · Harvard · PhD · 2025
Algorithms for Fair Redistricting - Anna Trella · Harvard · PhD · 2025
Deployable Online Reinforcement Learning Algorithms - Patsy Jammal · Georgia Tech · PhD · 2025
Digital Twin-Driven Condition Monitoring Approach for Aircraft Carbon Brakes - T Monahan · Oxford · PhD · 2025
Spatiotemporal tidal prediction and analysis through physics-informed machine learning - Mahmudul Bari Hridoy · Texas Tech · PhD · 2025
Infectious Disease Dynamics in Heterogeneous Populations: Stochastic, Seasonal, and Structured Epidemic Models with Intervention Strategies
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.