Library · Machine learning & AI
What Recurrent Neural Networks got wrong, from 38 theses
Summary written by a model from the records below: Recurrent neural networks frequently encounter training instability, compounding error accumulation across multi-step sequences, and performance deficits compared to tree-based ensembles or alternative architectures. Practitioners also reject recurrent models due to opaque internal states that hinder interpretability and difficulties maintaining accuracy under regularization or distribution shifts. These records come from graduate theses, 2021 to 2026, each labeled with its degree where the repository states it, and each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Plus 6 problems the authors left open, listed at the end.
Failure modes, by theses reporting them
Select a mode to jump to its records.
Recurrent neural networks suffer from vanishing and exploding gradients over long sequences
Standard recurrent architectures encounter severe gradient pathologies, vanishing gradients, and exploding gradients during backpropagation through time across extended horizons. These numerical instabilities prevent networks from retaining long-term dependencies, cause output fluctuations during dynamic tracking, and lead to divergence on stiff dynamics.
Tried and failed
Standard recurrent neural network applied to sequential degradation data modeling. Outcome: did not generalise. Reason: Captures only recent memory and fails to address long-term dependencies in the sequence
Advanced data-driven methods for prognostics and life extension of assets using condition monitoring and sensor data. · Cranfield · PhD
Considered and rejected
Considered and rejected: Rejected using Recurrent Neural Networks (RNNs) with backpropagation through time in GF due to numerical instability during higher-order hypergradient computation
On machine learning methods for time series with financial applications · Oxford · PhD
Considered and rejected
Considered and rejected: Rejected standard Recurrent Neural Networks (RNNs) due to vanishing gradient issues with long-term sequential dependencies in power load cycles.
IMPROVING CYBER RESILIENCE OF SHIPBOARD POWER SYSTEMS USING MACHINE LEARNING · Naval Postgraduate School · Master's
Tried and failed
Standard recurrent neural networks applied to long-term time series prediction. Reason: Gradient vanishing prevents retaining early sequence features over long horizons
Wind Power Prediction and Uncertainty Modeling for Power System Operation · Saskatchewan · PhD
Tried and failed
physics-informed neural networks and recurrent neural networks applied to stiff differential equations. Outcome: did not converge. Reason: severe gradient pathologies during training on stiff multiscale dynamics
Approximation of Large Stiff Acausal Models · MIT · PhD
Tried and failed
Vanilla recurrent neural network applied to Dynamic actuator system identification. Outcome: unstable. Reason: Vanishing and exploding gradients caused output fluctuations during periodic dynamic tracking
LSTM sequence-to-sequence based System Identification and feedforward-feedback control of piezoelectric actuators · Iowa State · Master's
Tried and failed
Training standard recurrent neural networks with backpropagation applied to long audio sequence recognition. Outcome: unstable. Reason: Exploding gradients during backpropagation through time over long sequences.
Biologically Inspired Spiking Neural Networks for Speech Recognition · EPFL · PhD
Considered and rejected
Considered and rejected: Rejected basic Simple Recurrent Neural Networks (RNNs) due to vanishing gradients that prevented learning long-range word dependencies across long sequences.
Data driven strategies for product design · UT Austin · PhD
Considered and rejected
Considered and rejected: Rejected Recurrent Neural Networks (RNNs) due to excessive resource consumption hindering real-time performance and vanishing gradient problems from frequent zero-velocity fixations
Leveraging the multi-modal nature of communication in immersive XR environments · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected feedforward neural networks (FFNN) and standard recurrent neural networks (Elman and Jordan RNNs) for hydrologic sequence modeling due to inability to retain sequential memory and vulnerability to the vanishing gradient problem.
Machine Learning Methods for Modeling Streamflow in Intermittent Rivers and Ephemeral Streams · Texas Tech · PhD
Considered and rejected
Considered and rejected: Rejected standard recurrent neural networks (RNNs) for Processing Cells in favor of LSTM cells to avoid the vanishing gradient problem over deep logical circuits.
Pushing the frontier of quantum many-body simulation using classical computers · Cornell · PhD
Recurrent neural networks underperform tree-based ensembles and simpler baselines on tabular time series
Across tabular sequential regression and classification tasks, recurrent models consistently fall behind gradient boosted trees, random forests, and simpler linear or support vector baselines. Limited sample sizes and tabular feature formats cause recurrent networks to suffer from high variance and wider confidence intervals without improving predictive accuracy.
Tried and failed
LSTM on sentiment and technical features applied to financial price direction prediction. Outcome: worse than baseline. Reason: Recurrent neural networks underperformed traditional support vector machines on multimodal tabular and sentiment features
Machine Learning for Financial Market Forecasting · Harvard · Master's
Tried and failed
ARIMA and recurrent neural networks applied to transient emission time series prediction. Outcome: worse than baseline. Reason: severe non-linearity and limited training sample size caused overfitting and high variance compared to gradient boosting
Tried and failed
recurrent neural networks for time-series regression applied to tabular time-series behavior forecasting. Outcome: worse than baseline. Reason: Deep sequence models underperformed gradient boosted trees on tabular sequential regression features.
Understanding and Predicting Sit-Stand Desk Usage Patterns and Willingness among Knowledge Workers: A Data-Driven Approach · Virginia Tech · Master's
Tried and failed
Stacked Bi-LSTM applied to time-series anomaly detection. Outcome: worse than baseline. Reason: Tree-based ensembles outperformed the recurrent neural network on the tabular sensor features.
Anomaly Detection and Prevention in Smart Grid Transmission System Using Machine Learning · Texas Tech · PhD
Tried and failed
recurrent neural networks for clinical trajectory modelling applied to short tabular longitudinal laboratory trajectories. Outcome: worse than baseline. Reason: failed to outperform simple logistic regression and exhibited substantially wider confidence intervals
Clinical risk prediction of progression to severe dengue illness during the febrile phase in primary healthcare settings · Imperial · PhD
Tried and failed
recurrent neural networks for tabular wear forecasting applied to component degradation time series regression. Outcome: worse than baseline. Reason: deep sequence models did not outperform tree-based regressors despite higher computational cost
Digital Twin-Driven Condition Monitoring Approach for Aircraft Carbon Brakes · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Rejected Recurrent Neural Network (RNN) approach for future streamflow modeling due to superior objective function metrics achieved by Random Forest
Climate change impact on the spatial distribution of droughts in Kirindi oya and Maduru oya dry zone river basins in Sri Lanka · University of Moratuwa · Master's
Autoregressive rollouts accumulate compounding errors over extended forecast horizons
Recurrent networks suffer from severe error accumulation and decaying correlation when generating multi-step predictions or long-term rollouts. Evaluating stateful models across iterative steps without resetting hidden states leads to fictitious history accumulation, worsening trajectory errors compared to empirical or feedforward baselines.
Tried and failed
recurrent neural networks for time-series forecasting applied to long-term reservoir production forecasting. Outcome: worse than baseline. Reason: Severe error accumulation and inability to capture long-term decline trends compared to standard empirical baselines
Asset Development and Sweet Spot Identification in Unconventional Reservoirs Using Machine Learning Approaches · Texas Tech · PhD
Tried and failed
autoregressive LSTM for multi-step time series forecasting applied to daily aggregated emergency department arrivals. Outcome: worse than baseline. Reason: fine-grained recurrent models accumulate errors when aggregating over longer daily horizons compared to short hourly predictions
Predicting the Unpredictable: Comparing Statistical Forecasting and Deep Learning Models for Forecasting Emergency Department Arrivals · Harvard · Master's
Tried and failed
recurrent neural network with convolutional layers applied to spatiotemporal fluid flow PDE dynamics. Outcome: did not generalise. Reason: fails to capture source patterns/location after initial steps and suffers decaying correlation over time
Deep Learning for Dynamical Systems: Modeling, Prediction, and Control · Georgia Tech · PhD
Tried and failed
recurrent neural networks for sequential decision making applied to error recovery action suggestion. Outcome: worse than baseline. Reason: human error accumulation over sequential steps degraded accuracy and robustness relative to non-recurrent models
Facilitating Reliable Autonomy with Human-Robot Interaction · Georgia Tech · PhD
Lost to a baseline
Recurrent neural networks (RNNs) produced worse long-term rollout position and orientation errors than feedforward MLPs on stiff contact datasets despite lower single-step error.
Tried and failed
stateful recurrent neural networks in iterative solvers applied to coupled nonlinear equilibrium problems. Outcome: did not converge. Reason: evaluating sequence models across optimization steps without resetting hidden states causes fictitious history accumulation and divergence
Physics enabled Data-driven structural analysis for mechanical components and assemblies · Georgia Tech · PhD
Recurrent architectures are outperformed by feedforward networks, convolutional models, and transformers
Simpler feedforward networks, convolutional neural networks, and transformer architectures often surpass recurrent models in predictive accuracy and computational efficiency. Practitioners reject recurrent networks due to high state buffering overheads, inordinate training times, and architectural constraints that limit spatial feature extraction.
Tried and failed
recurrent and time-delay neural networks applied to aerodynamic time-series surrogate modeling. Outcome: worse than baseline. Reason: training sensitivity caused recurrent architectures to underperform simpler feedforward networks on time-series data
A Controller Development Methodology Incorporating Unsteady, Coupled Aerodynamics and Flight Control Modeling for Atmospheric Entry Vehicles · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Rejected Recurrent Neural Networks (RNNs) because they use a single weight matrix across units, limiting spatial feature detection and requiring inordinate training time compared to CNNs.
Predicting Phenotypes From Novel Genomic Markers Using Deep Learning · Saskatchewan · Master's
Tried and failed
recurrent neural networks applied to time-series trigger classification. Outcome: worse than baseline. Reason: simpler feedforward neural network performed better or was preferred for classification
Three Terrestrial Planets Transiting Mid-to-Late M Dwarfs and Constraints on the Occurrence of Such Worlds · Harvard · PhD
Considered and rejected
Considered and rejected: BiLSTM-CRF architectures were rejected for named entity recognition in favor of BERT-based models because transformer-based models consistently outperform sequential recurrent neural networks.
A pipeline for data and knowledge extraction from material science literature to accelerate scientific discovery · Georgia Tech · PhD
Tried and failed
recurrent neural networks applied to hardware branch prediction. Outcome: worse than baseline. Reason: yielded lower prediction accuracy and high inference-engine state buffering overheads compared to CNN architectures
Using convolutional neural networks to improve branch prediction · UT Austin · PhD
Recurrent models fail to generalize across distribution shifts and specialized data structures
Recurrent networks struggle when evaluating modern transformer outputs, noise-vocoded speech, and zero-inflated intermittent series. Continuous distribution assumptions produce invalid negative values on zero-heavy data, while acoustic degradation and physically similar sensor fault signatures cause severe misclassification.
Tried and failed
continuous recurrent neural network regression applied to zero-inflated intermittent time series. Outcome: did not generalise. Reason: assuming a single continuous distribution failed to predict true zeros and generated invalid negative values
Machine Learning Methods for Modeling Streamflow in Intermittent Rivers and Ephemeral Streams · Texas Tech · PhD
Tried and failed
recurrent neural network text classifier applied to transformer-generated synthetic text detection. Outcome: did not generalise. Reason: architectures trained on older generation models fail to detect modern transformer-based outputs
Defending Against Misuse of Synthetic Media: Characterizing Real-world Challenges and Building Robust Defenses · Virginia Tech · Doctorate
Tried and failed
neuro-inspired feature extraction with recurrent networks applied to speech recognition under acoustic degradation. Outcome: did not generalise. Reason: models degraded on noise-vocoded speech but tolerated periodic speech, contradicting human perceptual robustness
Features of hearing: applications of machine learning to uncover the building blocks of hearing · Imperial · PhD
Tried and failed
recurrent neural network for multi-class anomaly classification applied to cyber-physical system sensor incident classification. Outcome: did not generalise. Reason: Failed to distinguish between physically similar fault signatures of missing vs corrupted sensor readings
H2OGAN: A Deep Learning Approach for Detecting and Generating Cyber-Physical Anomalies · Virginia Tech · Master's
Opaque internal states lead practitioners to reject recurrent networks for lack of interpretability
Hidden states in recurrent neural networks are opaque and make it difficult to explain multivariate predictions. Authors rejected recurrent models in favor of Gaussian mixture hidden Markov models and traditional time series methods to maintain system transparency.
Considered and rejected
Considered and rejected: Rejected sequence models (Gated Recurrent Units / GRU) and SDEs for temperature, HVAC consumption, and electricity spot price processes in favor of traditional time series models (SARIMA, Holt-Winters, ARX-GJR-GARCH) for superior interpretability and efficiency.
Three Essays on Energy Markets · Johns Hopkins · PhD
Considered and rejected
Considered and rejected: Rejected recurrent neural networks (LSTMs) for temporal modeling due to opaque hidden states lacking interpretability for multivariate data, choosing Gaussian mixture HMMs instead
Introducing Productive Engagement for Social Robots Supporting Learning · EPFL · PhD
Considered and rejected
Considered and rejected: Rejected using recurrent networks to learn change detection metrics because it reduces model interpretability.
Detecting and Leveraging Changes in Temporal Data · Georgia Tech · PhD
Regularization techniques and feature additions degrade recurrent network performance
Applying dropout regularization to recurrent models failed to reduce complexity in stacked multi-layer networks and degraded generalization on large training datasets. Similarly, incorporating redundant sensor features caused performance drops rather than improving trajectory forecasts.
Tried and failed
Dropout regularization in deep multi-layer LSTM networks applied to time series imputation. Outcome: worse than baseline. Reason: Failed to mitigate added complexity and error inflation caused by stacking multiple recurrent layers
Data Driven Early Stage Design Support for Offshore Wind Farms · University College Dublin · PhD
Tried and failed
applying dropout regularization to recurrent neural networks applied to time series prediction on large datasets. Outcome: worse than baseline. Reason: regularization degraded performance without improving generalization due to the sufficiently large training dataset size
Traffic Signal Phase and Timing Prediction: A Machine Learning and Controller Logic Hybrid Approach · Virginia Tech · Doctorate
Tried and failed
adding redundant sensor features to recurrent networks applied to kinematic trajectory prediction. Outcome: worse than baseline. Reason: additional sensor features introduced redundancy that caused performance degradation or offered no improvement
Gait Phase Estimation and Foot Trajectory Prediction During Dynamic Walking Using Gated Recurrent Units · Virginia Tech · Master's
Left open by the authors
Problems the authors named and did not get to.
Left open
Benchmark and compare the dual temporal attention recurrent model against time-series Transformers for many-to-many sequence prediction. Blocker: None
Recurrence and Temporal Attention Synergy for Optimal Time-Series Modeling and Interpretability · Texas State University · PhD
Left open
Implement recurrent architectures like LSTMs using historical sequential sensor data for RUL estimation and anomaly reconstruction. Blocker: None
Toward The Democratization of Industrial Machine Learning · Georgia Tech · PhD
Left open
Develop a robust training method for recurrent neural networks processing static images with both recurrence and internal noise. Blocker: None
EXPLAINING FEATURES OF SIMPLE HUMAN DECISIONS USING BAYESIAN NEURAL NETWORKS · Georgia Tech · PhD
Left open
Add a recurrent decoder or sliding attention mechanism over sequential window embeddings to incorporate temporal context in stereo-EEG seizure detection. Blocker: Requires clinical stereo-EEG dataset with sub-second channel-level seizure annotations used in the thesis
Channel-Level Seizure Onset and Offset Detection in Stereo-EEG with Sub-Second Temporal Precision Using Machine Learning · Harvard · Master's
Left open
Develop recurrent neural networks or RNN-based GANs to map time-based geomechanical oilfield data. Blocker: No specific architecture, dataset, or performance targets are specified
Enhanced Oil Field Data-Wrangling using Machine Learning · Texas Tech · PhD
Left open
Train an LSTM-based recurrent neural network to automate coarse visual categorization of astronomical seed light curves. Blocker: None
ExoSpotter: Few Shot Relevance Feedback For Learning High Recall Exoplanet Search · MIT
Who has worked on this
Authors of the public theses cited on this page, most-cited first.
- Farhang Forghanparast · Texas Tech · PhD · 2023
Machine Learning Methods for Modeling Streamflow in Intermittent Rivers and Ephemeral Streams - Grant James Bell · Harvard · Master's · 2026
Channel-Level Seizure Onset and Offset Detection in Stereo-EEG with Sub-Second Temporal Precision Using Machine Learning - Rajendra Shrestha · Texas Tech · PhD · 2026
Anomaly Detection and Prevention in Smart Grid Transmission System Using Machine Learning - Yiqing Zhou · Cornell · PhD · 2025
Pushing the frontier of quantum many-body simulation using classical computers - Riccardo Bovo · Imperial · PhD · 2025
Leveraging the multi-modal nature of communication in immersive XR environments - Patsy Jammal · Georgia Tech · PhD · 2025
Digital Twin-Driven Condition Monitoring Approach for Aircraft Carbon Brakes - Ruocheng Yin · Iowa State · Master's · 2025
LSTM sequence-to-sequence based System Identification and feedforward-feedback control of piezoelectric actuators - Jung Hoon Chung · Virginia Tech · Master's · 2025
Understanding and Predicting Sit-Stand Desk Usage Patterns and Willingness among Knowledge Workers: A Data-Driven Approach - Gohar Shoukat · University College Dublin · PhD · 2025
Data Driven Early Stage Design Support for Offshore Wind Farms - Hayley Wisman · Virginia Tech · Master's · 2025
Gait Phase Estimation and Foot Trajectory Prediction During Dynamic Walking Using Gated Recurrent Units - Justin Andrews · Texas Tech · PhD · 2025
Enhanced Oil Field Data-Wrangling using Machine Learning - Nauman Ahad · Georgia Tech · PhD · 2024
Detecting and Leveraging Changes in Temporal Data
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.