An Experimental Evaluation of Large Language Models in Supporting the DEX Multi-Criteria Decision-Making Process
Abstract
We experimentally assessed the capabilities of two mainstream artificial intelligence chatbots, ChatGPT and DeepSeek, to support the multi-criteria decision-making process. Specifically, we focused on using the method DEX (Decision EXpert) and investigated their performance in all stages of DEX model development and utilization. The results indicate that these tools may substantially contribute in the difficult stages of collecting and structuring decision criteria, and collecting data about decision alternatives. However, at the current stage of development, the support for the whole multi-criteria decision-making process is still lacking, mainly due to occasionally inconsistent and erroneous execution of methodological steps. To leverage the strengths of both approaches, we also propose a hybrid workflow for DEX model development that begins in the LLM and continues in the specialized DEXiWin software.
Full Text:
PDFReferences
Bana e Costa, C., De Corte, J.-M., Vansnick, J.-C. (2003): MACBETH (Overview of MACBETH multicriteria decision analysis approach). International Journal of Information Technology & Decision Making 11:02, 359–387. https://doi.org/10.1142/S0219622012400068.
Banerjee, S., Agarwal, A., Singla, S. (2024). LLMs will always hallucinate, and we need to live with this. https://doi.org/10.48550/arXiv.2409.05746.
Bansal, A. (2024): Comprehensive Study on LLM Agents. International Journal of All Research Education & Scientific Methods 12(8):2455-6211.
Bohanec, M., Rajkovič, V., Bratko, I., Zupan, B., Žnidaršič, M. (2013): DEX methodology: Three decades of qualitative multi-attribute modelling. Informatica 37, 49–54.
Bohanec, M. (2022): DEX (Decision EXpert): A qualitative hierarchical multi-criteria method. In: Multiple Criteria Decision Making (ed. Kulkarni, A.J.), Studies in Systems, Decision and Control 407, Singapore: Springer, https://doi.org/10.1007/978-981-16-7414-3_3.
Bohanec, M. (2024): DEXiWin: DEX Decision Modeling Software, User’s Manual, Version 1.2. Ljubljana: Institut Jožef Stefan, Delovno poročilo IJS DP-14747. https://dex.ijs.si/dexisuite/dexiwin.html.
Bohanec, M. (2025): Using large language models for creating, modifying and interpreting decision tables in multi-criteria method DEX. CECIIS 2025, 36th Central European Conference on Information and Intelligent Systems, Varaždin, Croatia, 651–658.
Bohanec, M., Rajkovič, U., Rajkovič, V (2025): Utilizing large language models for supporting multi-criteria decision modelling method DEX. IS 2025, Proc. of the 28th International Multiconference Information Society, Volume A, Ljubljana, Slovenia. 19–22.
Bray, T., Paoli, J., Sperberg, C.M., Maler, E., Yergeau, F. (2008): Extensible Markup Language (XML) 1.0 (Fifth Edition). https://www.w3.org/TR/REC-xml/.
Hämäläinen, R.P., Luoma, J., Saarinen, E. (Eds.). (2016): Behavioral Operational Research: A Capabilities Approach. Cham: Palgrave Macmillan. https://doi.org/10.1007/978-3-030-25405-6.
Ishizaka, A., Nemery, P. (2013): Multi-criteria decision analysis: Methods and software. Chichester: Wiley. ISBN: 978-1-119-97407-9.
Kamath, U., Keenan, K., Somers, G., Sorenson, S. (2024): Large Language Models: A Deep Dive: Bridging Theory and Practice. Springer, 506p, ISBN-13 978-3031656460.
Kulkarni, A.J. (Ed.) (2022): Multiple Criteria Decision Making. Studies in Systems, Decision and Control 407, Singapore: Springer, https://doi.org/10.1007/978-981-16-7414-3.
Radovanović, S., Delibašić, B., Vukanović, S. (2024): Combining LLM and DIDEX method to predict Internal Migrations in Serbia. Proc. 24th International Conference on Group Decision and Negotiation & 10th International Conference on Decision Support System Technology (GDN ICDSST 2024), Vol. 1: Technology as a support tool (Eds. S.P. Duarte, P. Zaraté, A. Lobo, B. Delibašić, T. Wachowicz, M.C. Ferreira), University of Porto.
Saaty, T.L., Vargas, L.G. (2012): Models, Methods, Concepts & Applications of the Analytic Hierarchy Process. Springer, US, New York, https://doi.org/10.1007/978-1-4614-3597-6.
Svoboda, I., Lande, D. (2024): Enhancing multi-criteria decision analysis with AI: Integrating analytic hierarchy process and GPT-4 for automated decision support. https://arxiv.org/abs/2402.07404.
Wang, H., Zhang, F., Mu, C. (2025). One for all: A general framework of LLMs-based multi-criteria decision making on human expert level. arXiv e-prints, 2502.15778. https://arxiv.org/abs/2502.15778.
Wang, L., Ma, C., Feng, X. et al. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 186345. https://doi.org/10.1007/s11704-024-40231-1.
Yuan, W., Liu, P., Gallé, M (2024): LLMCRIT: Teaching large language models to use criteria. Findings of the Association for Computational Linguistics: ACL 2024, 7929–7960, https://doi.org/10.18653/v1/2024.findings-acl.472.
DOI: https://doi.org/10.31449/inf.v49i4.12884
This work is licensed under a Creative Commons Attribution 3.0 License.