Entropy-Guided Assessment of Image Retrieval Systems: Advancing Grouped Precision as an Evaluation Measure for Relevant Retrievability

Tahar Gherbi, Ahmed Zeggari, Zianou Ahmed Seghir, Fella Hachouf

Abstract


The performance evaluation of Content Based Image Retrieval systems (CBIR), can be considered as a challenging and overriding problem even for human and expert users regarding the important numbers of CBIR systems proposed in the literature and applied to different image databases. The automatic measures widely used to assess CBIR systems are inspired from the general Text Retrieval (TR) domain such as precision and recall metrics. This paper proposes a new quantitative measure adapted to the CBIR particularity of relevant images grouping, which is based on the entropy of the returned relevant images. The proposed performance measure is easy to understand and to implement. A good discriminative power of the proposed measure is shown through a comparative study with the existing and well-known CBIR evaluation measures.

Full Text:

PDF

References


Yong Rui and Thomas S. Huang (1999) Image retrieval: Current techniques, promising directions and open issues, Journal of Visual Communication and Image Representation, v.10, pp. 39–62.

Smeulders, Arnold W. M. and Worring, Marcel and Santini, Simone and Gupta, Amarnath and Jain, Ramesh (2000) Content-Based Image Retrieval at the End of the Early Years, IEEE Trans. Pattern Anal. Mach. Intell., IEEE Computer Society, v.22(12), pp. 1349–1380.

Datta, Ritendra and Joshi, Dhiraj and Li, Jia and Wang, James Z. (2008), Image Retrieval: Ideas, Influences, and Trends of the New Age, ACM Comput. Surv., ACM Publisher, v.40(2), pp. 5:1– 5:60.

url = http://trec.nist.gov/.

Wu, Shengli and McClean, Sally (2006) Information Retrieval Evaluation with Partial Relevance Judgment, Flexible and Efficient Information Handling, Springer Berlin Heidelberg, pp.86–93.

Yong Rui and Huang, Thomas S (2000) Optimizing learning in image retrieval, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, v.1, pp. 236–243.

Müller, Henning and Müller, Wolfgang and Squire, David McG. and Marchand-Maillet, Stephane and Pun, Thierry (2001) Performance Evaluation in Content-based Image Retrieval: Overview and Proposals, Pattern Recogn. Lett. Elsevier Science Inc., v.22(5), pp.593–601.

Cooper, William S. (1973) On selecting a measure of retrieval effectiveness, Journal of the American Society for Information Science, v.24(2), pp. 87-100.

Lewis, David D. (1995) Evaluating and Optimizing Autonomous Text Classification Systems, Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, Seattle, Washington, USA, pp.246–254.

Buckley, Chris and Voorhees, Ellen M. (2000) Evaluating Evaluation Measure Stability,Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieva, ACM, Athens, Greece, pp. 33–40.

David D. Lewis (1995) The TREC-4 Filtering Track, TREC, National Institute of Standards and Technology (NIST), Special Publication 500-236.

Zhao, Ying (2005) Criterion Functions for Document Clustering, phd thesis University of Minnesota USA.

Rosenberg, Andrew and Hirschberg, Julia (2007) V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pp.410–420.

Salton, G. (1971) The SMART Retrieval System—Experiments in Automatic DocumentProcessing, Prentice-Hall, Inc., Upper Saddle River, NJ, USA.

C. J. van Rijsbergen (1979) Information Retrieval, Butterworth. [16] MPEG-7 (2000) Subjective evaluation of the MPEG-7 retrieval accuracy measure (ANMRR)./WG11, Doc. M6029.

Savvas A. Chatzichristofis and Chryssanthi Iakovidou and Yiannis S. Boutalis and Elli Angelopoulou (2014) Mean Normalized Retrieval Order (MNRO): a new content-based image retrieval performance measure, Multimedia Tools Appl., v.70(3), pp.1767–1798.

Harald Kosch and Paul Maier (2010) Content-Based Image Retrieval Systems - Reviewing and Benchmarking, JDIM, v.8(1), pp.54–64.

Gareth Loy and Jan-Olof Eklundh (2005) A Review of Benchmarking Content Based Image Retrieval.

Henning Müller, Antoine Geissbuhler, Stephane Marchand Maillet, Paul Clough (2004) Benchmarking image retrieval applications, Workshop on Visual Information Systems, pp.334–337.

Dubey, Shiv Ram (2020) A Decade Survey of Content Based Image Retrieval using Deep Learning, IEEE Transactions on Circuits and Systems for Video Technology

Xiwu Zhang and Lei Wang and Yan Su (2021) Visual place recognition: A survey from deep learning perspective, Pattern Recognition, v.113(2), pp. 107760.

Alistair Moffat and Justin Zobel (2008) Rank-biased precision for measurement of retrieval ef- fectiveness, ACM Trans. Inf. Syst., v.27, pp.2:1- 2:27.




DOI: https://doi.org/10.31449/inf.v47i7.4661

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.