Chapter Four · failure evidence
What Agent-Based & Multi-Agent Simulation got wrong, from 59 dissertations
The records document technical limitations and modeling hurdles encountered when developing agent-based and multi-agent simulation systems. Researchers frequently face computational bottlenecks at scale, calibration discrepancies against empirical data, and policy degradation in complex multi-agent learning environments. These records come from PhD theses at 20 institutions, 2021 to 2026. Each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Prohibitive computational scaling and communication overhead hinder large agent populations
Large-scale agent simulations and centralized multi-agent training frequently encounter exponential computational scaling and prohibitive communication bottlenecks. As a result, researchers are forced to abandon fine-grained agent representations in favor of aggregated spatial models or decentralized approximations.
Tried and failed
standard belief propagation on joint distribution applied to multi-commodity network routing problems. Outcome: infeasible cost. Reason: computational complexity scales exponentially with the number of agents or paths
Advances in Algorithms for Sampling, Optimization and Inference in Disordered Systems · EPFL
Tried and failed
iterative greedy backward heuristic for bin packing applied to dynamic multi-hub parcel consolidation. Outcome: too slow. Reason: computational scaling limitations when executed within an agent-based simulation of large real-size networks
Hyperconnected Parcel Logistics: Planning and Assessment · Georgia Tech
Considered and rejected
Considered and rejected: Rejected standard Agent-Based Modeling (ABM) and detailed game-theoretic models for large-scale mathematical network optimization due to prohibitive computational complexity and scaling limitations.
Integrating Optimisation Models and Behavioural Insights for Effective Transport Evacuation Strategies · Research Repository UCD
Considered and rejected
Considered and rejected: Centralized training for large-scale MARL traffic networks rejected due to excessive communication overhead, coordination complexity, and computational bottlenecks across thousands of agents.
Adaptive intelligent traffic control systems in smart city · Cranfield
Considered and rejected
Considered and rejected: Rejected Agent-Based Simulation as the primary baseline due to excessive computational resource demands for broad design-space exploration sweeps.
A METHODOLOGY FOR THE MODULARIZATION OF OPERATIONAL SCENARIOS FOR MODELLING AND SIMULATION · Georgia Tech
Considered and rejected
Considered and rejected: Rejected a fully scaled agent-based model of monarch butterfly migration due to excessive computational demands in NetLogo, choosing a generalized unitless pinch-point migration model instead.
Behavioral and Geophysical Factors Influencing Success in Long Distance Navigation · DukeSpace
Considered and rejected
Considered and rejected: Rejected agent-based mobility simulations in favor of grid-based raster modeling due to computational and large-scale data constraints across Europe.
Considered and rejected
Considered and rejected: Rejected treating drivers and customers at an individual agent-based level during vehicle rebalancing optimization due to computational tractability, preserving regional spatial aggregation instead
Towards A Robust Integrated Urban Mobility System: Public Transit and Ride-Sharing Systems · MIT
Considered and rejected
Considered and rejected: Agent-Based Simulation (ABS) and System Dynamics (SD) rejected; ABS added excessive computational demand and SD lacked discrete event scheduling variability
Tried and failed
nonparametric simulation-based iterative dynamic programming applied to heterogeneous-agent equilibrium models. Outcome: too slow. Reason: linear aggregate dynamics without complex market clearing make parametric approximations computationally faster
Considered and rejected
Considered and rejected: Rejected centralized global multi-agent MDP optimization due to exponential computational complexity; adopted decentralized inter-agent Gaussian reward scaling
Hybrid Sensor Networks for Active Monitoring: Collaboration, Optimization, And Resilience · Georgia Tech
Flawed behavioral and decision rules fail to capture complex emergent dynamics
Modeling agents with uncoupled interactions, rigid rule sets, or static choice sets leads to severe discrepancies in collective system dynamics. These simplified behavioral assumptions cause agents to fail at multi-objective trade-offs, miscalculate spatial relocation, or become erratic outside narrow operating conditions.
Tried and failed
independent uncoupled agent contraction modeling applied to bulk volume reduction of active networks. Reason: neglecting collective structural transmission and coupling drastically underestimated macroscopic network contraction magnitude
Mesoscale Modeling and Probing of Blood Clot Contraction · Georgia Tech
Tried and failed
agent-based social cognitive modeling applied to predicting emergent group structure metrics. Outcome: no signal. Reason: mean follower self-schemas lacked predictive power for network density or centralization across interactions
Using Agent-Based Modeling to Test and Integrate Process-Oriented Perspectives of Leadership Emergence · Virginia Tech
Tried and failed
Linear conflict count extrapolation models applied to multi-agent trajectory management system performance. Outcome: no signal. Reason: Oversimplified geometrical interaction models failed to capture complex emergent dynamics of decoupled 4D trajectory planning
Conceptual-Level Analysis and Design of Unmanned Air Traffic Management Systems · Georgia Tech
Tried and failed
adaptive fuzzy rule-based agent microsimulation applied to multi-objective budget allocation. Reason: agents failed to balance competing objectives simultaneously, consistently neglecting at least one budget
An adaptive agent-based multicriteria simulation system · Cranfield
Tried and failed
Regret aversion modeling applied to spatial worker relocation decisions. Outcome: did not generalise. Reason: Wrongly predicted over-movement from distant rather than nearby agents under spatial information sharing
Information Sharing and Operational Transparency on On-Demand Service Platforms · Georgia Tech
Tried and failed
inverting agent causal beliefs in coupled simulations applied to coupled predator-prey policy dynamics. Outcome: no signal. Reason: preference calculations and intrinsic dynamic damping buffered against changes in agent belief polarity
Exploring policy change through agent-based simulation · EPFL
Tried and failed
sentiment-driven persona conditioning in multi-agent systems applied to multi-agent deliberative debate. Outcome: worse than baseline. Reason: lacked persuasive reasoning and negotiation ability
Toward Deliberative AI: Multi-Agent LLMs for Real-World Reasoning · Virginia Tech
Tried and failed
counterfactual simulation with fixed choice sets applied to mechanism design policy evaluation. Outcome: did not generalise. Reason: agents strategically adjust choice lists in response to policy changes rather than holding preferences fixed
Tried and failed
deterministic rule-based modeling of dynamic systems applied to agent survival simulation. Outcome: did not generalise. Reason: Failed to survive and became erratic outside a narrow payoff range
An adaptive agent-based multicriteria simulation system · Cranfield
Considered and rejected
Considered and rejected: Rejected assuming optimal decision-making by individual agents in separate compartments, adding random noise instead to reflect realistic behavior.
Artificial Intelligence-Based Approaches for Analysis and Optimization of Complex Systems: Case Studies in Computational Epidemiology · Scholarship at UWindsor Institutional Repository
Structural miscalibration and parameters borrowed from literature fail to generalize
Calibrating micro-level agent models using literature-reported values or macro-level aggregate statistics often washes out critical decision logic. This leads to simulated velocity distributions, welfare directions, and consumption patterns that conflict with empirical observations across heterogeneous groups.
Tried and failed
calibrating structural model with literature-borrowed parameters applied to multi-agent network welfare analysis. Outcome: did not generalise. Reason: borrowed parameters produced contradictory welfare directions across heterogeneous groups
Essays in macroeconomics · UT Austin
Tried and failed
microscopic agent-based flow simulation calibration applied to urban traffic speed profile prediction. Reason: Simulated midblock velocity distributions statistically differed from observed empirical measurements across scenarios
Evaluation of simulation model vehicle activity output for traditional and modal emission modeling · Iowa State
Tried and failed
exponential discounting heterogeneous-agent modeling applied to empirical consumption-wealth sensitivity to policy. Outcome: did not generalise. Reason: produced response coefficients significantly outside empirical confidence intervals
Tried and failed
two-asset heterogeneous-agent incomplete-markets modeling applied to household consumption-savings distribution. Outcome: did not generalise. Reason: incorrectly predicts extreme, persistent marginal propensities to consume across constrained and unconstrained agent groups
Tried and failed
using literature-reported interaction parameters directly applied to agent-based trajectory simulation calibration. Outcome: did not converge. Reason: literature values failed to generalize to the specific open-field environmental dynamics
Video-Based Parameter Calibration and Virtual Simulation of Pedestrian Crowd Dynamics · Virginia Tech
Tried and failed
macro-level statistical calibration of agent-based models applied to agent decision-making dynamics. Outcome: no signal. Reason: optimising solely for aggregate macro outputs washed out the impact of micro-level decision logic
Agent-based modelling of people movement using mixed methods approach · Imperial
Tried and failed
agent-based modeling with empirically derived parameters applied to healthcare access disparities by geographic distance. Outcome: did not generalise. Reason: remoteness did not act as a uniform barrier, with long-distance agents exhibiting higher utilization rates
Tried and failed
approximate aggregation using parametric forecasting rules applied to heterogeneous agent macroeconomic models. Outcome: did not generalise. Reason: lack of general equilibrium price smoothing caused forecasting rules to fit poorly
Considered and rejected
Considered and rejected: Rejected agent-based modeling for policy simulation because the early-stage deployment of blockchain-DPP systems lacked granular empirical behavioral data at the individual stakeholder level.
Optimization of Reverse Supply Chain For End-of-life Products · Cranfield
Multi-agent reinforcement learning degrades under partial observability and non-stationarity
Multi-agent learning algorithms and opponent modeling struggle to maintain performance when faced with non-stationary dynamics and deceptive adversaries. Under partial observability or decorrelated channels, policies fail to converge or diverge significantly from cooperative human play styles.
Tried and failed
single agent opponent modeling applied to asymmetric multiagent games with deceptive adversaries. Outcome: overfit. Reason: overfits to simple training behaviors, failing to detect deceptive opponents during evaluation
Robust and Scalable Multiagent Reinforcement Learning in Adversarial Scenarios · MIT
Tried and failed
robust optimization in multi-agent game theory applied to autonomous vehicle decision making. Reason: Failed to find optimal solutions due to lacking probabilistic modeling of opponent reasoning uncertainty.
Socially Compatible Behavior Prediction, Decision-making, and Control for Autonomous Vehicles in Mixed Traffic · Texas Tech
Tried and failed
penalty-based Q-learning applied to multi-agent reinforcement learning under decorrelated fading. Outcome: unstable. Reason: encountered performance degradation during extended training under completely decorrelated channel conditions
Training native intelligent communication systems · Imperial
Tried and failed
Multi-Agent Deep Deterministic Policy Gradient applied to multi-robot decentralized sequential decision-making. Outcome: worse than baseline. Reason: struggled with partial observability and coordination under uncertainty despite extensive hyperparameter tuning
Lost to a baseline
In multi-agent reinforcement learning with partial observations, the centralized policy optimization algorithm with partial observations slightly underperformed the centralized algorithm with full observations because policies learned with partial observations are a subset of those learned with full observations.
A NEW ZEROTH-ORDER ORACLE FOR DISTRIBUTED AND NON-STATIONARY LEARNING · DukeSpace
Considered and rejected
Considered and rejected: Rejected relying on single-agent RL applied to multiple agents because non-stationary environmental dynamics prevent convergence to optimal policies.
Leveraging Information Sharing for Satellite Navigation and Coordination · MIT
Tried and failed
unregularized reinforcement learning applied to human-agent multi-agent coordination. Outcome: did not generalise. Reason: agents diverged from human play styles, reducing action prediction accuracy and cooperation
Building Strategic AI Agents for Human-centric Multi-agent Systems · MIT
Tried and failed
adversarial inverse reinforcement learning with heterogeneous data applied to agent trajectory modeling across behavioral regimes. Outcome: worse than baseline. Reason: mixing data from distinct behavioral regimes degraded policy learning and prediction accuracy
Considered and rejected
Considered and rejected: Rejected treating multi-agent learning with equal priority or assuming the opponent is a stationary environment (simultaneous gradient descent-ascent in MADDPG) in favor of hierarchical Stackelberg modeling.
Certifiable Algorithms for Reinforcement Learning: Safety-Critical and Game-Theoretic Perspectives · ResearchWorks
Complex multi-agent coordination is frequently outperformed by simple decentralized heuristics
Elaborate tactical coordination and game-theoretic optimization are often overwhelmed by simple greedy heuristics in dense or latency-constrained settings. Furthermore, overestimating opponent reasoning or enforcing coordination under low stochasticity can introduce deadlocks and traffic turbulence.
Tried and failed
higher-order cognitive hierarchy modeling applied to multi-agent bounded rationality games. Outcome: worse than baseline. Reason: Overestimating opponent reasoning depth yields suboptimal responses against lower-level strategic agents.
Control and game-theoretic methods for secure cyber-physical-human systems · Georgia Tech
Tried and failed
game-theoretic decentralized multi-agent coordination applied to mixed-autonomy traffic bottleneck merging. Outcome: worse than baseline. Reason: Low penetration rates created localized coordination friction and turbulence under high congestion.
Enhancing Freeway Merge Section Operations via Vehicle Connectivity · Virginia Tech
Tried and failed
Derivative-free continuous relaxation for task allocation applied to multi-agent routing and scheduling. Outcome: worse than baseline. Reason: Greedy path planning dominates makespan when task-to-agent ratio is large, limiting optimization gains
Distributed Heterogeneous Multi-robot Task Allocation in Communication-limited Environments · Georgia Tech
Tried and failed
explicit tactical coordination in multi-agent reinforcement learning applied to high-density swarm adversarial engagements. Outcome: worse than baseline. Reason: explicit multi-agent coordination was overwhelmed and outscored by decentralized greedy baseline policies in dense combat
Coordinating Team Tactics for Swarm-vs.-Swarm Adversarial Games · Georgia Tech
Tried and failed
optimal control strategy under latency applied to multi-agent pursuit-evasion games. Outcome: worse than baseline. Reason: computational delay in calculating optimal intercept points degraded performance below simple heuristics
Perception-Action Loop for Multi-Robot Systems with Deep Learning · Penn
Tried and failed
optimal search path coordination applied to multi-agent resource extraction. Outcome: no signal. Reason: optimal scheduling yielded statistically insignificant utility gains over a simple heuristic
Measuring the Communicative Constitution of Partial Organizations as Complex Systems · Virginia Tech
Tried and failed
decentralized multi-agent planning under low communication stochasticity applied to multi-robot inspection coordination. Outcome: worse than baseline. Reason: Low communication dropouts caused multi-agent deadlocks, whereas higher stochasticity helped break symmetrical decision deadlocks.
Sequential decision-making and active sensing for autonomous outer space operations · UT Austin
Inappropriate granularity leads to the rejection of agent-based modeling for aggregate frameworks
Agent-based modeling was rejected in several applications because tracking individual behaviors introduced unnecessary operational noise without capturing system-level dynamics. Researchers instead preferred system dynamics, discrete event simulation, or network analysis to model macro feedback loops and queue structures.
Considered and rejected
Considered and rejected: Rejected Discrete Event Simulation and Agent-Based Modelling for the primary employee performance framework because individual random human behaviors cannot be modeled operationally, choosing System Dynamics for policy-level abstraction.
Employee performance modelling using system dynamics · Cranfield
Considered and rejected
Considered and rejected: Rejected agent-based modeling (ABM) in favor of system dynamics causal mapping and stock-flow modeling because food supply chain flows and institutional structural accumulations were better captured at aggregate levels.
Food Justice Transitions in Buffalo, NY: The Role of Values-Based Approaches in Social Movements · DSpace at SUNY Buffalo
Considered and rejected
Considered and rejected: Rejected using System Dynamics (SD) or Agent-Based (AB) simulation, choosing Discrete Event Simulation (DES) to accurately capture discrete timestamped tasks and stochastic queue dynamics.
Modeling of Urban Freight Deliveries; Operational Performance at the Final 50 Feet · ResearchWorks
Considered and rejected
Considered and rejected: Rejected discrete-event simulation (DES) and agent-based modeling (ABM) in favor of System Dynamics because DES and ABM focus on individual event occurrences rather than macro strategic feedback complexity.
A dynamic analysis of the success factors that affect performance measurement implementation · Texas Tech
Considered and rejected
Considered and rejected: Rejected discrete-event simulation (DES) and agent-based modelling (ABM) for ecosystem topological analysis in favor of network science for representing static complex network structures.
Analysis of the evolution of aerospace manufacturing ecosystems · Cranfield
Poorly tuned traffic penalty and routing heuristics trigger congestion and non-convergence
Routing algorithms that apply heavy traffic penalties or traffic-following weights cause excessive detours in sparse networks and gridlock across alternative paths in dense settings. Additionally, removing travel mode constraints causes agents without transport alternatives to induce non-convergence in iterative assignment.
Tried and failed
Traffic-following path planning applied to multi-agent decentralized routing. Outcome: worse than baseline. Reason: High traffic-following weight caused excessive and unnecessary detours in low-density regimes
Nature-inspired self-organization in autonomous multi-agent systems · UT Austin
Tried and failed
high traffic-penalty weighting in routing algorithms applied to dense multi-agent path planning. Outcome: worse than baseline. Reason: capacity saturation occurred because routing lacked trajectory prediction, causing congestion across alternative paths
Nature-inspired self-organization in autonomous multi-agent systems · UT Austin
Tried and failed
relaxing baseline travel mode accessibility constraints applied to iterative multimodal dynamic network equilibrium simulation. Outcome: did not converge. Reason: agents without viable transport alternatives caused unrealistic behaviour and non-convergence in iterative assignment
Simulation and optimisation of dynamic multimodal traffic network with shared mobility systems · Imperial
Left open by the authors
Problems the authors named and did not get to.
Left open
Develop an agent-based behavioral simulation model to simulate pedestrian dynamics under varying thermal comfort conditions in urban spaces. Blocker: Lacks specific behavioral parameters, empirical calibration datasets, and formal agent decision-rule specifications.
Urban comfort and connectivity in hot-climate cities: evaluating public space usability with the UCCI framework in Riyadh · University of Nottingham Repository
Left open
Model bacteria-substrate surface interactions using agent-based discrete element simulations to study CaCO3 grain morphologies. Blocker: Lacks specific interaction parameters and formulation details for the proposed agent-based discrete element model.
Left open
Incorporate agent-based simulation and stochastic optimization analysis tools into the circular remanufacturing decision framework. Blocker: Lack of specific modeling requirements, parameter distributions, agent behaviors, or mathematical formulations
Decision-Making Framework for Circular Economy in Remanufacturing · Scholarship at UWindsor Institutional Repository
Left open
Develop automated or inverse generative AI methods to parameterise agent logic in agent-based models of people movement. Blocker: Lacks specific target architectures, parameter formulations, and validation benchmarks
Agent-based modelling of people movement using mixed methods approach · Imperial
Left open
Implement the Agent sub-model (Model A) to manage natural language representations, perception, and composite agent decision performance in SAbMDE. Blocker: Lacks specific formal design specifications and clear evaluation targets for Model A
Development Cycle Modeling and Risk Calculation · Texas Tech
Left open
Couple integrated transit time distribution hydrologic models with agent-based models of human behavior to simulate watershed salt application scenarios. Blocker: Lacks specific target watershed, behavioral rules, agent-based formulation, or performance targets
Unveiling Causal Links, Temporal Patterns, and System-Level Dynamics of Freshwater Salinization Using Transit Time Distribution Theory · Virginia Tech
Left open
Benchmark the performance of the COVID-19 transfer learning framework against agent-based epidemiological models. Blocker: None
Social Network Analysis: A Machine Learning Approach · Scholarship at UWindsor Institutional Repository
Left open
Validate individual operational assumptions used to formalize and reconcile policy process theories within the agent-based simulation model. Blocker: No specific empirical datasets, experimental designs, or validation criteria are provided for the theoretical assumptions
Exploring policy change through agent-based simulation · EPFL
Left open
Formulate stochastic extensions of compartmental biocontrol models and evaluate them on alternative plant-pathogen-biocontrol agent systems. Blocker: Lacks specific target systems, experimental datasets, or concrete mathematical formulation for stochasticity.
Using Modelling to Optimise the Use of Biological Control Agents Against Soil-Borne Plant Pathogens · Cambridge
Left open
Develop a formal model for how rational agents commensurate credence and depth of ingression when evaluating paradox premises. Blocker: Lacks specific mathematical framework, target metric, or concrete formal constraints for the commensuration mechanism
God and other enigmas · UT Austin
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.