Machine Bias: A Survey of Issues

Ana Farič, Ivan Bratko

Abstract


Some recent applications of Artificial Intelligence, particularly machine learning, have been strongly criticised in general media and professional literature. Applications in domains of justice, employment and banking are often mentioned in this respect. The main critic is that these applications are biased with respect to so called protected attributes, such as race, gender and age. The most notorious example is the system COMPAS which is still in use in the American justice system despite severe criticism. The aim of our paper is to analyse the trends of discussion about bias in machine learning algorithms using the COMPAS as an example. The main problem we observed is that even in the field of AI, there is no generally agreed upon definition of bias which would enable operational use in preventing bias. Our conclusions are that (1) improved general education concerning AI is needed to enable better understanding of AI methods in everyday applications, and (2) better technical methods must be developed for reliably implementing generally accepted societal values such as equality and fairness in AI systems.


Full Text:

PDF

References


Alelyani, S. (2021). Detection and Evaluation of Machine Learning Bias. Applied Sciences, 11(14). https://doi.org/10.3390/app11146271.

Angwin, A., Larson, J., Mattu, J. & Kirchner, L. (2016). Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And it’s Biased Against Blacks. ProPublica.

Artificial Intelligence Act, European Parliament, 14 June 2023; current updated unofficial version January 2024.

Berk, R., Heidari, H., Jabbari, S., Kearns, M & Roth, A. (2021) Fairness in Criminal Justice Risk Assessments: The State of the Art. Sociological Methods & Research, 50(1), 3-44. https://doi.org/10.1177/0049124118782533.

Blanzeisky, W. & Cunningham, P. (2021). Algorithmic Factors Influencing Bias in Machine Learning. arXiv preprint. https://doi.org/10.48550/arXiv.2104.14014.

Cestnik, B., Kononenko, I. & Bratko, I. (1987). ASSISTANT 86: A knowledge-elicitation tool for sophisticated users: In: Bratko, I., Lavrač, N. (eds) Progress in Machine Learning: Proc. of European Working Session on Learning EWSL 87. Sigma Press, 1987, 31-45.

Chakraborty, J., Majumder, J. & Menzies, T. (2021). Bias in Machine Learning Software: Why? How? What to do? arXiv preprint. https://doi.org/10.48550/arXiv.2105.12195.

Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R. & Goel, S. (2018). The Measure and Mismeasure of Fairness. arXiv preprint. https://doi.org/10.48550/arXiv.1808.00023.

Courtland, R. (2018). Bias detectives: the researchers striving to make algorithms fair. Nature, 558, 357-360. https://doi.org/10.1038/d41586-018-05469-3.

Dastin, J. (11.10.2018). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G.

Dressel, J. & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1).

Flores, A. W., Bechtel, K. & Lowenkamp, C. T. (2016). False Positives, False Negatives, and False Analyses: A Rejoinder to “Machine Bias: There’s Software Used Across the Country to Predict Future Criminals And It’s Biased Against Blacks.” Federal Probation Journal, 80(2).

Gordon, D. F. & Desjardins, M. (1995). Evaluation and Selection of Biases in Machine Learning. Machine Learning, 20, 5-22. https://doi.org/10.1023/A:1022630017346.

Hardt, M., Price, E. & Srebro, N. (2016). Equality of Opportunity in Supervised Learning. arXiv preprint. https://doi.org/10.48550/arXiv.1610.02413.

Hellström, T., Dignum, V. & Bensch, S. (2020). Bias in Machine Learning – What is it Good for? arXiv preprint. https://doi.org/10.48550/arXiv.2004.00686.

Holsinger, A. M., Lowenkamp, C. T., Latessa, E. J., Serin, R., Cohen, T. H., Robinson, C. R., Flores, A. W. & vanBenschoten, S. W. (2018). A Rejoinder to Dressel and Farid: New Study Finds Computer Algorithm Is More Accurate Than Humans at Predicting Arrest and as Good as a Group of 20 Lay Experts. Federal Probation Journal, 82(2), 51-56.

Hüllermeier, E., Fober, T. & Mernberger, M. (2013). Inductive Bias. Encyclopedia of Systems Biology. https://doi.org/10.1007/978-1-4419-9863-7_927.

Kleinberg, J., Mullainathsn, S. & Raghavan, M. (2016). Inherent Trade-Offs in the Fair Determination of Risk Scores. arXiv preprint. https://doi.org/10.48550/arXiv.1609.05807.

Levin, S. (8.9.2016). A beauty contest was judged by AI and the robots didn’t like dark skin. The Guardian. https://www.theguardian.com/technology/2016/sep/08/artificial-intelligence-beauty-contest-doesnt-like-black-people.

Mehrabi, A., Morstatter, F., Saxena, N., Lerman, K. & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6), 1-35. https://doi.org/10.48550/arXiv.1908.09635.

Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M. E. … Staab, S. (2020). Bias in data-driven artificial intelligence systems – An introductory survey. WIREs Data Mining and Knowledge Discovery, 10(3). https://doi.org/10.1002/widm.1356.

Rudin, C. (2019). Stop explaining black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead. Nature Machine Intelligence, 1(5), 206-215. https://doi.org/10.1038/s42256-019-0048-x.

Spielkamp, M. (2017). Inspecting Algorithms for Bias. MIT Technology Review, July 2017.

Staff, R. (7.12.2016). New Zealand passport robot tells applicant of Asian descent to open eyes. Reuters. https://www.reuters.com/article/us-newzealand-passport-error-

idUSKBN13W0RL.

Sun, O., Nasroui, O. & Shafto, P. (2020). Evolution and impact of bias in human and machine learning algorithms interaction. PLoS ONE, 15(18). https://doi.org/10.1371/journal.pone.0235502.

UNESCO Recommendation on the Ethics of Artificial Intelligence, 2021. https://unesdoc.unesco.org/ark:/48223/pf0000381137.

Yu, H., Shen, Z., Miao, C., Lesser, V. R. & Yang, Q. (2018). Building Ethics into Artificial Intelligence. arXiv preprint. https://doi.org/10.48550/arXiv.1812.02953.




DOI: https://doi.org/10.31449/inf.v48i2.5971

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.