[Paper Review] Comparing simulation output accuracy of discrete event and agent based models: a quantitative approach

Mazlina Abdul Majid, Uwe Aickelin+1 more|arXiv (Cornell University)|Jul 13, 2009

Complex Systems and Decision Making17 references20 citations

TL;DR

This study compares discrete event simulation (DES) and agent-based simulation (ABS) for modeling human reactive behavior in retail operations, using a UK department store's fitting room as a case study. Both methods demonstrated equivalent output accuracy in simulating staff and customer behavior, showing equal potential for evaluating management policy changes.

ABSTRACT

In our research we investigate the output accuracy of discrete event simulation models and agent based simulation models when studying human centric complex systems. In this paper we focus on human reactive behaviour as it is possible in both modelling approaches to implement human reactive behaviour in the model by using standard methods. As a case study we have chosen the retail sector, and here in particular the operations of the fitting room in the women wear department of a large UK department store. In our case study we looked at ways of determining the efficiency of implementing new management policies for the fitting room operation through modelling the reactive behaviour of staff and customers of the department. First, we have carried out a validation experiment in which we compared the results from our models to the performance of the real system. This experiment also allowed us to establish differences in output accuracy between the two modelling methods. In a second step a multi-scenario experiment was carried out to study the behaviour of the models when they are used for the purpose of operational improvement. Overall we have found that for our case study example both, discrete event simulation and agent based simulation have the same potential to support the investigation into the efficiency of implementing new management policies.

Motivation & Objective

• To evaluate the output accuracy of discrete event simulation (DES) and agent-based simulation (ABS) in modeling human reactive behavior.
• To assess the effectiveness of both modeling approaches in supporting operational improvement through policy simulation.
• To validate model outputs against real-world performance in a retail fitting room setting.
• To compare the two methods in a controlled, multi-scenario experiment for policy impact analysis.

Proposed method

• A real-world validation experiment was conducted to compare model outputs against actual system performance in a UK department store's women's wear fitting room.
• Reactive behavior of customers and staff was modeled using standard methods in both DES and ABS frameworks.
• A multi-scenario simulation experiment was performed to evaluate the models' behavior under different management policy implementations.
• Model accuracy was assessed by comparing simulated outcomes to observed real system performance.
• Both DES and ABS were implemented with identical behavioral logic to ensure fair comparison.

Experimental results

Research questions

• RQ1 How do discrete event simulation and agent-based simulation compare in modeling human reactive behavior in a retail setting?
• RQ2 What is the output accuracy of DES and ABS when validated against real system performance?
• RQ3 Can both modeling approaches effectively support operational improvement through policy simulation?
• RQ4 How do the models perform under different management policy scenarios?

Key findings

• Both discrete event simulation and agent-based simulation achieved equivalent output accuracy when modeling human reactive behavior in the fitting room.
• The validation experiment confirmed that both models accurately reflected the performance of the real system.
• The multi-scenario experiment demonstrated that both approaches are equally effective for evaluating management policy impacts.
• No significant differences in model output accuracy were observed between DES and ABS under the same behavioral assumptions.