Library · Machine learning & AI
What Rule-Based & Expert Systems got wrong, from 29 theses
Summary written by a model from the records below: The records demonstrate that rule-based and expert systems frequently struggle when dealing with dynamic environments, complex linguistic variability, and noisy human behavioral data. Across numerous tasks, handcrafted heuristics and rigid deterministic thresholds underperformed probabilistic models, learned representations, and standard statistical baselines. These records come from graduate theses, 2021 to 2026, each labeled with its degree where the repository states it, and each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Plus 3 problems the authors left open, listed at the end.
Failure modes, by theses reporting them
Select a mode to jump to its records.
Rule-based and heuristic natural language processing fails to handle complex linguistic variation
Handcrafted syntactic patterns, regular expressions, and grammatical rules frequently break down when faced with complex negation, layout variations, and idiosyncratic semantic shifts. Consequently, these rigid text processing approaches yield low precision and poor recall compared to learned models, or distort underlying sentence semantics during data manipulation.
Tried and failed
rule-based natural language processing applied to clinical report diagnosis extraction. Reason: failed to correctly handle complex clinical negation, causing poor specificity and low F1 score
Artificial intelligence applications for the acquisition, analysis and reporting of cardiovascular magnetic resonance imaging · Imperial · PhD
Tried and failed
rule-based pattern matching NLP applied to inter-parameter dependency extraction from documentation. Outcome: no signal. Reason: rigid syntactic patterns failed to match diverse real-world natural language documentation structures
Effective Automation of Black-Box Testing for REST APIs with Machine Learning and Language Models · Georgia Tech · PhD
Tried and failed
rule-based text data augmentation applied to intent classification. Outcome: worse than baseline. Reason: heuristic word-level perturbations alter sentence semantics and label alignment
Lifelong Machine Learning with Data Efficiency and Knowledge Retention · EPFL · PhD
Tried and failed
rule-based compositional semantics and derivational morphology applied to compound participle meaning prediction. Outcome: did not generalise. Reason: rules failed to capture idiosyncratic semantic shifts and semi-productivity patterns
The semantics of past participles · UT Austin · PhD
Tried and failed
rule-based dependency parser with candidate scoring applied to natural language to formal specification translation. Reason: candidate completions lacking matching input categories were overly penalized by the negative scoring mechanism
Framework for Automatic Translation of Hardware Specifications Written in English to a Formal Language · Virginia Tech · Doctorate
Lost to a baseline
Rule-based baseline NLP (EasyCIE_GUI) achieved an F1 of only 0.43 and precision of 0.34 (at 0.59 recall), losing substantially to deep learning models (BiLSTM F1 0.64–0.66, precision 0.62–0.67).
Surgical Site Infection (SSI) Identification Across Multiple Facilities and Surgery Types Using Multimodal Data and Deep Learning · University of Washington · PhD
Considered and rejected
Considered and rejected: Rejected using only plain rule-based parsing in ChemDataExtractor due to brittle failure on minor grammatical variations and low inorganic recall (56%).
Magnetic and Superconducting Materials Discovery: Employing Data Science, Natural Language Processing and Machine Learning · Cambridge · PhD
Considered and rejected
Considered and rejected: Rejected pure rule-based comparative adjective/adverb parsing due to the complexity of building grammar rules for varied linguistic structures.
Compa: A Comparative Retrieval and Analytical Engine for Consumer Products · Texas State University · Master's
Rule-based deduction engines and heuristic abstractions lack adaptability and learning capacity
Heuristic deduction engines and rule models cannot learn from new observations, demanding extensive manual reprogramming when requirements change. As a result, synthetic rule sets fail to generalize to real operational errors and heuristic abstractions produce oversimplified models or non-minimal state spaces.
Considered and rejected
Considered and rejected: Rejected automatically distinguishing between genuinely unknown biochemical gaps and reconstruction errors in dead-end tests due to poor accuracy of previous automated heuristics.
The fundamentals of genome-scale metabolic models and their application to the study of evolution and cancer · Boston University · PhD
Considered and rejected
Considered and rejected: Rejected synthetic rule-based error injection models because their error distributions failed to generalize to real generation errors
Fine-grained evaluation for text summarization · UT Austin · PhD
Tried and failed
counterexample-guided abstraction refinement applied to partial-information game synthesis. Outcome: worse than baseline. Reason: automated refinement heuristics produce non-minimal abstract state spaces compared to manual domain-specific abstractions
Assured decison-making for autonomous systems · UT Austin · PhD
Considered and rejected
Considered and rejected: Default high dependency thresholds in the Flexible Heuristics Miner algorithm were rejected because they led to oversimplified process models in low-structured domains.
Rezeptions- und Interpretationsprozesse von Lehrpersonen bei datengestützten Entscheidungen. Exploration und Förderung · Tübingen · PhD
Considered and rejected
Considered and rejected: Rejected rule-based deduction engines (e.g. from IoIF) because semantic rules cannot learn from new observations or scale across new problem domains.
MM-ADM: A model-based approach to multidisciplinary design to support automated decision-making · Georgia Tech · PhD
Considered and rejected
Considered and rejected: Rejected simple confidence-based early-stopping heuristics and generic upweighting methods for error mitigation because they fail to capture consistent, human-understandable failure modes.
A Data-Based Perspective on Model Reliability · MIT · PhD
Rigid mathematical thresholds and deterministic rules fail in dynamic or continuous environments
Hardcoded thresholds and deterministic switching rules cannot accommodate dynamic workloads, continuous variables, or boundary conditions fairly. When deployed in non-stationary or long-horizon tasks, these inflexible policies become erratic, generate false positives, and fail outside narrow operating ranges.
Considered and rejected
Considered and rejected: Rejected rule-based heuristics and MPC for long-term operational optimization due to lack of adaptability in dynamic environments and poor scaling over long-horizon stochastic tasks.
Tried and failed
static heuristic-based threshold switching rules applied to dynamic architectural protocol selection. Outcome: worse than baseline. Reason: simple thresholds fail to capture complex workload dynamics, yielding incorrect decisions in half the cases
AI-DRIVEN ADAPTIVE DISTRIBUTED SYSTEMS IN UNTRUSTED ENVIRONMENTS · Penn · PhD
Considered and rejected
Considered and rejected: Crisp rule-based expert systems using rigid mathematical thresholds were rejected because they fail to handle boundary cases fairly and create overly complex, unmaintainable rule bases.
Strategie działania inteligentnych systemów wspierających kształcenie operujące na danych nieprecyzyjnych Strategies of operation for intelligent tutoring systems operating on imprecise data · Adam Mickiewicz University · PhD
Considered and rejected
Considered and rejected: Fixing static μ thresholds rule-based was rejected in favor of dynamically low-pass filtering synaptic weight efficacy
Building new memories using the past: Interplay between semantic and episodic memory in brains and machines · Imperial · PhD
Considered and rejected
Considered and rejected: Rejected classical rule-based conditional programming (hardcoded angle thresholds) because it produced false positives while moving/pretending to shoot and could not generalize across diverse shooting styles.
Live Perception and Real Time Motion Prediction with Deep Neural Networks and Machine Learning · Harvard · Master's
Deterministic heuristics fail to capture uncertainty and stochasticity in human behavior
Rule-based models fall short when estimating human activity or responses because deterministic logic ignores baseline behavioral stochasticity and sensor noise. Furthermore, crowdsourced or manual rule sets encode shallow heuristics that fail on complex edge cases and slow down users without boosting decision accuracy.
Tried and failed
rule-based behavior inference applied to human response behavior under disruption. Reason: produced systematic overestimation by ignoring baseline stochasticity in human behavior
Toward a Resilient Public Transportation System: Effective Monitoring and Control under Service Disruptions · MIT · PhD
Tried and failed
heuristic training for counterfactual forecasting applied to decision making under unrepresentative baselines. Outcome: did not generalise. Reason: training failed when faced with unrepresentative baseline distributions or prospective conditional framing
FAST AND FRUGAL STATISTICAL HEURISTICS: TRANSFER OF COUNTERFACTUAL FORECASTING TRAINING WITHIN AND ACROSS DOMAINS · Penn · PhD
Tried and failed
deterministic rule-based logical models applied to human activity estimation. Outcome: worse than baseline. Reason: strict logical rules cannot handle noise and uncertainty compared to probabilistic modeling
Cognitive Human Activity and Plan Recognition for Human-Robot Collaboration · MIT · PhD
Tried and failed
extracting decision rules from human-voted behavior applied to sequential decision-making in disrupted environments. Outcome: did not generalise. Reason: crowdsourced rules encoded shallow, suboptimal heuristics with plateauing performance gains
Managing The Gig Economy Via Behavioral And Operational Lenses · Penn · PhD
Tried and failed
rule-based heuristic decision support system applied to human decision-making under degraded information. Outcome: worse than baseline. Reason: increased user response times without improving accuracy, especially for lower-performing users
Understanding and Supporting Decision Making in Denied and Degraded Environments · Georgia Tech · PhD
Tried and failed
rule-based anti-pattern heuristics for manual diagnosis applied to identifying algorithmic complexity vulnerabilities. Reason: developers could not accurately diagnose complex edge cases using manual pattern rules
Theory and Patterns for Avoiding Regex Denial of Service · Virginia Tech · Master's
Rule-based classifiers and heuristic feature selectors underperform probabilistic and learned baselines
Predefined rules and heuristic filtering struggle to model continuous attributes and complex topological relationships, leading to high false positive rates and unnecessary constraints. Across benchmark tasks like gait identification and investment classification, rule-based algorithms consistently lost to probabilistic methods and decision trees.
Tried and failed
heuristic rule-based constraint extraction applied to circuit symmetry constraint detection. Outcome: worse than baseline. Reason: produced higher false positive rates and generated unnecessary constraints compared to learned representations
Layout automation for custom integrated circuits · UT Austin · PhD
Lost to a baseline
Heuristic rules-based algorithm achieved lower overall gait event identification accuracy across locomotion modes (94.87%) compared to the unsupervised BP-AR-HMM (99.6%).
Machine Learning and Wearable Sensors for the Estimation of Biomechanical Variables Outside the Laboratory · University of Oregon · PhD
Lost to a baseline
Rule-based PART classifier (0.5005 accuracy on investment cost) lost to C4.5 Trees J48 (0.571 average accuracy) and Naive Bayes (0.5091 accuracy on investment cost)
Renewable Energy Communities: A Preference Learning Approach to Evaluate Differential Participation in Cities · Politecnico di Torino · PhD
Tried and failed
heuristic filter-based feature selection applied to transient stability classification. Outcome: worse than baseline. Reason: standard filters failed to capture complex causal topological interactions compared to Markov blanket selection
Topological changes in data-driven dynamic security assessment for power system control · Imperial · PhD
Left open by the authors
Problems the authors named and did not get to.
Left open
Extend the tree-based rule extraction heuristic to estimate continuous variables and generalize across different problem sizes. Blocker: Lack of specific target continuous variables, validation metrics, or concrete generalization methodology
A Machine Learning-Based Heuristic to Explain Game-Theoretic Models · Virginia Tech · Master's
Left open
Test whether fast-and-frugal counterfactual forecasting heuristics transfer to real-world scenarios lacking accessible ground truth. Blocker: No specific real-world domain, evaluation methodology, or benchmark dataset is defined for scenarios lacking ground truth
FAST AND FRUGAL STATISTICAL HEURISTICS: TRANSFER OF COUNTERFACTUAL FORECASTING TRAINING WITHIN AND ACROSS DOMAINS · Penn · PhD
Left open
Adapt the rule-extraction and hierarchical decision tree heuristic to explain policy functions in reinforcement learning models. Blocker: The proposal is an exploratory direction without a specified target RL environment, policy model, or evaluation benchmark.
A Machine Learning-Based Heuristic to Explain Game-Theoretic Models · Virginia Tech · Master's
Who has worked on this
Authors of the public theses cited on this page, most-cited first.
- Avinashh Kumar Baswapuram · Virginia Tech · Master's · 2024
A Machine Learning-Based Heuristic to Explain Game-Theoretic Models - Anne Lederer Duke · Penn · PhD · 2023
FAST AND FRUGAL STATISTICAL HEURISTICS: TRANSFER OF COUNTERFACTUAL FORECASTING TRAINING WITHIN AND ACROSS DOMAINS - Rashmi Ravishankar · MIT · PhD · 2026
Edge Computing in Space: Design Optimization and Reinforcement Learning Scheduling of Onboard Computing Satellites - Albert Albesa-González · Imperial · PhD · 2025
Building new memories using the past: Interplay between semantic and episodic memory in brains and machines - Myeongsoo Kim · Georgia Tech · PhD · 2025
Effective Automation of Black-Box Testing for REST APIs with Machine Learning and Language Models - Devlin Moyer · Boston University · PhD · 2025
The fundamentals of genome-scale metabolic models and their application to the study of evolution and cancer - Marcin Szczepański · Adam Mickiewicz University · PhD · 2025
Strategie działania inteligentnych systemów wspierających kształcenie operujące na danych nieprecyzyjnych Strategies of operation for intelligent tutoring systems operating on imprecise data - GIULIO CAVANA · Politecnico di Torino · PhD · 2025
Renewable Energy Communities: A Preference Learning Approach to Evaluate Differential Participation in Cities - Saachi Jain · MIT · PhD · 2024
A Data-Based Perspective on Model Reliability - Arjun Chakraborty · University of Washington · PhD · 2024
Surgical Site Infection (SSI) Identification Across Multiple Facilities and Surgery Types Using Multimodal Data and Deep Learning - Sarah Damaris Bez · Tübingen · PhD · 2024
Rezeptions- und Interpretationsprozesse von Lehrpersonen bei datengestützten Entscheidungen. Exploration und Förderung - Chenyuan Wu · Penn · PhD · 2024
AI-DRIVEN ADAPTIVE DISTRIBUTED SYSTEMS IN UNTRUSTED ENVIRONMENTS
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.