IDedupNet: A MobileNetV3-Based Deep Learning Framework for Efficient Image Deduplication in Cloud Computing Environments

Mohd Hasan Mohiuddin, Latha Tamilselvan

Abstract


Image deduplication is becoming increasingly important for cloud storage infrastructures to handle the increasing amount of multimedia material. Through increased storage efficiency, effective picture deduplication may optimize resources and save expenses. It also improves performance by facilitating quicker access, utilizing less bandwidth, and enhancing data integrity. Although heuristic-based classical deduplication techniques work well in various storage infrastructures, they cannot keep up with the dynamic nature of cloud storage resources. This study presents IDedupNet, a revolutionary DL-based framework that improves infrastructure performance in cloud computing by efficiently detecting duplicate and near-duplicate photos. Our approach uses deep learning for picture encoding and deduplication, allowing it to manage duplicate photos in highly dynamic contexts efficiently. Additionally, we provide a Learning-Based Image Deduplication (LBID) approach that improves deduplication performance by extending the use of the IDedupNet model. Our suggested deep learning model has considerable advantages over current models, regularly beating them and obtaining a high accuracy of 98.68% on benchmark datasets, which increases the model's credibility. The underlying technique and deep learning framework may be easily integrated into real-time cloud storage systems to increase customer satisfaction and infrastructure efficiency.


Full Text:

PDF

References


Amdewar Godavari, Chapram Sudhakar, T. Ramesh. (2024). Hybrid deduplication system with content-based cache for cloud environment. Elsevier. 36(5), pp.1-12. https://doi.org/10.1016/j.jksuci.2024.102030.

NANNAN ZHAO, MUHUI LIN, HADEEL ALBAHAR, ARNAB K. PAUL, ZHIJIE HUANG, SUBIL ABRAHAM, USA KEREN CHEN, VASILY TARASOV, DIMITRIOS SKOURTIS, ALI ANWAR AND ALI R. BUTT. (2024). An End-to-End High-Performance Deduplication Scheme for Docker Registries and Docker Container Storage Systems. ACM, pp.1-33. https://doi.org/10.1145/3643819

S. Usharani and K. Dhanalakshmi. (2023). An image storage duplication detection method using recurrent learning for smart application services. Springer. 79, pp.1-27. https://doi.org/10.1007/s11227-023-05042-4

Nagappan Mageshkumar, J. Swapna, A. Pandiaraj, R. Rajakumar, Moez Krichen, and Vinayakumar Ravi. (2023). Hybrid cloud storage system with enhanced multilayer cryptosystem for secure deduplication in cloud. Elsevier. 4, pp.301-309. https://doi.org/10.1016/j.ijin.2023.11.001

Muhammad Atta Othman Ahmed, Ibrahim A. Abbas and Yasser AbdelSatar. (2023). HDSNE a new unsupervised multiple image database fusion learning algorithm with fexible and crispy production of one database: a proof case study of lung infection diagnose In chest X-ray images. Springer. 23, pp.1-15. https://doi.org/10.1186/s12880-023-01078-3

Paul Baker and Luke Collins. (2023). Creating and analysing a multimodal corpus of news texts with Google Cloud Vision's automatic image tagger. Elsevier. 3(1), pp.1-10. https://doi.org/10.1016/j.acorp.2023.100043

Ch. Prathima, Naresh Babu Muppalaneni, and K. G. Kharade. (2022). Deduplication of IoT Data in Cloud Storage. Springer, pp.147-157. https://doi.org/10.1007/978-981-16-5090-1_13

K. Pragash and J. Jayabharathy. (2022). A survey on DE – Duplication schemes in cloud servers for secured data analysis in various applications. Elsevier. 24, pp.1-6. https://doi.org/10.1016/j.measen.2022.100463

K. Vijayalakshmi and V. Jayalakshmi; (2021). Analysis on data deduplication techniques of storage of big data in cloud . 2021 5th International Conference on Computing Methodologies and Communication (ICCMC). http://doi:10.1109/iccmc51019.2021.9418445

Kwabena, Owusu-Agyemeng; Qin, Zhen; Zhuang, Tianming and Qin, Zhiguang (2019). MSCryptoNet: Multi-Scheme privacy-preserving deep learning in cloud computing. IEEE Access, 7, 29344 - 29354. http://doi:10.1109/ACCESS.2019.2901219

Zheng, Xiaoyu; Zhou, Yuyang; Ye, Yalan and Li, Fagen (2019). A cloud data deduplication scheme based on certificateless proxy re-encryption. Journal of Systems Architecture, 102, pp.1-44. http://doi:10.1016/j.sysarc.2019.101666

Yinjin Fu; Nong Xiao; Tao Chen and Jian Wang; (2021). Fog-to-MultiCloud Cooperative eHealth Data Management with Application-Aware Secure Deduplication . IEEE Transactions on Dependable and Secure Computing. http://doi:10.1109/tdsc.2021.3086089

Yunling Wang; Meixia Miao; Jianfeng Wang and Xuefeng Zhang; (2021). Secure deduplication with efficient user revocation in cloud storage. Computer Standards & Interfaces. 78, pp.1-8. http://doi:10.1016/j.csi.2021.103523

Guipeng Zhang; Zhenguo Yang; Haoran Xie and Wenyin Liu; (2021). A secure authorized deduplication scheme for cloud data based on blockchain. Information Processing & Management. http://doi:10.1016/j.ipm.2021.102510

Xu, Guangping; Tang, Bo; Lu, Hongli; Yu, Quan and Sung, Chi Wan (2019). LIPA: A Learning-based Indexing and Prefetching Approach for Data Deduplication. 35th Symposium on Mass Storage Systems and Technologies (MSST), pp.299–310. http://doi:10.1109/msst.2019.00010

Jia, Wei; Li, Li; Li, Zhu; Zhao, Shuai and Liu, Shan (2020). Scalable Hash From Triplet Loss Feature Aggregation For Video De-duplication. Journal of Visual Communication and Image Representation, 72, pp.1-9. http://doi:10.1016/j.jvcir.2020.102908

Zhou, Zhili; Yang, Ching-Nung; Kim, Cheonshik and Cimato, Stelvio (2020). Introduction to the special issue on deep learning for real-time information hiding and forensics. Journal of Real-Time Image Processing, 17, pp.1-5. http://doi:10.1007/s11554-020-00947-2

Rajput, Amitesh Singh; Raman, Balasubramanian and Imran, Javed (2020). Privacy-preserving human action recognition as a remote cloud service using RGB-D sensors and deep CNN. Expert Systems with Applications, 152, pp.1-15. http://doi:10.1016/j.eswa.2020.113349

Anuradha, M.; Jayasankar, T.; Prakash, N.B.; Sikkandar, Mohamed Yacin; Hemalakshmi, G.R.; Bharatiraja, C. and Britto, A. Sagai Francis (2020). IoT enabled Cancer Prediction System to Enhance the Authentication and Security using Cloud Computing. Microprocessors and Microsystems, 103301–. http://doi:10.1016/j.micpro.2020.103301

Magesh Kumar S; Balasundaram A; Kothandaraman D; Auxilia Osvin Nancy V; P. J. Sathish Kumar and Ashokkumar S; (2020). An Approach to Secure Capacity Optimization in Cloud Computing using Cryptographic Hash Function and Data De-duplication . 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS). http://doi:10.1109/iciss49785.2020.9315892

Amna Asif; Shaheen Khatoon; Md Maruf Hasan; Majed A. Alshamari; Sherif Abdou; Khaled Mostafa Elsayed and Mohsen Rashwan; (2021). Automatic analysis of social media images to identify disaster type and infer appropriate emergency response . Journal of Big Data. 8(83), pp.1-28. http://doi:10.1186/s40537-021-00471-5

Takeshita, Jonathan; Karl, Ryan and Jung, Taeho (2020). [IEEE 2020 29th International Conference on Computer Communications and Networks (ICCCN) - Honolulu, HI, USA (2020.8.3-2020.8.6)] 2020 29th International Conference on Computer Communications and Networks (ICCCN) - Secure Single-Server Nearly-Identical Image Deduplication. 1–6. http://doi:10.1109/icccn49398.2020.9209728

K. Vijayalakshmi and V. Jayalakshmi; (2021). Analysis on data deduplication techniques of storage of big data in cloud . 2021 5th International Conference on Computing Methodologies and Communication (ICCMC). http://doi:10.1109/iccmc51019.2021.9418445

Manish Shetty; Chetan Bansal; Sumit Kumar; Nikitha Rao; Nachiappan Nagappan and Thomas Zimmermann; (2021). Neural Knowledge Extraction From Cloud Service Incidents . 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). http://doi:10.1109/icse-seip52600.2021.00031

Miao Zhang; Fangxin Wang; Yifei Zhu; Jiangchuan Liu and Zhi Wang; (2021). Towards cloud-edge collaborative online video analytics with fine-grained serverless pipelines. Proceedings of the 12th ACM Multimedia Systems Conference. http://doi:10.1145/3458305.3463377

Lu, Zhigang; Wu, Yuewen; Xu, Jiwei and Wang, Tao (2019). An Acceleration Method for Docker Image Update. IEEE International Conference on Fog Computing (ICFC), 15–23. http://doi:10.1109/ICFC.2019.00010

Zhang, Meng (2020). E-Commerce Comment Sentiment Classification Based on Deep Learning. IEEE 5th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA) .pp.184–187. http://doi:10.1109/ICCCBDA49378.2020.9095734

Li, Xuan; Li, Jin; Yiu, Siuming; Gao, Chongzhi and Xiong, Jinbo (2019). Privacy-preserving edge-assisted image retrieval and classification in IoT. Frontiers of Computer Science. http://doi:10.1007/s11704-018-8067-z

Tengfei Xing; Yang Gu; Zhichao Song; Zhihui Wang; Yiping Meng; Nan Ma; Pengfei Xu; Runbo Hu and Hua Chai; (2019). A Traffic Sign Discovery Driven System for Traffic Rule Updating . Proceedings of the 3rd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery. Pp. 52–55 http://doi:10.1145/3356471.3365237

Prince Hamandawana; Awais Khan; Jongik Kim and Tae-Sun Chung; (2021). Accelerating ML/DL Applications with Hierarchical Caching on Deduplication Storage Clusters . IEEE Transactions on Big Data. 8(6), pp. 1622 - 1636 http://doi:10.1109/tbdata.2021.3106345

Wang, H., Tobon V., D. P., Hossain, M. S., & Saddik, A. E. (2021). Deep Learning (DL)-Enabled System for Emotional Big Data. IEEE Access, 9, 116073–116082. http://doi:10.1109/access.2021.3103501

Andrew Boutros; Mathew Hall; Nicolas Papernot and Vaughn Betz; (2020). Neighbors From Hell: Voltage Attacks Against Deep Learning Accelerators on Multi-Tenant FPGAs. 2020 International Conference on Field-Programmable Technology (ICFPT). http://doi:10.1109/icfpt51103.2020.00023

Jia, Wei; Li, Li; Li, Zhu; Zhao, Shuai and Liu, Shan (2020). Triplet Loss Feature Aggregation for Scalable Hash. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp.1918–1922. http://doi:10.1109/icassp40776.2020.9053908

Jansen, Christoph; Annuscheit, Jonas; Schilling, Bruno; Strohmenger, Klaus; Witt, Michael; Bartusch, Felix; Herta, Christian; Hufnagl, Peter and Krefting, Dagmar (2020). Curious Containers: A framework for computational reproducibility in life sciences with support for Deep Learning applications. Future Generation Computer Systems, 112, 209–227. http://doi:10.1016/j.future.2020.05.007

Du, Xiaoyu; Le, Quan and Scanlon, Mark (2020). International Conference on Cyber Security and Protection of Digital Services (Cyber Security) - Automated Artefact Relevancy Determination from Artefact Metadata and Associated Timeline Events. 1–8. http://doi:10.1109/CyberSecurity49315.2020.9138874

Chen, Hong (2020). International Conference on Electronics and Sustainable Communication Systems (ICESC) - Big Data Cleaning Algorithm based on Repetitive Change Detection and GANs. 477–480. http://doi:10.1109/ICESC48915.2020.9155958

Abuhasel, Khaled Ali and Khan, Mohammad Ayoub (2020). A Secure Industrial Internet of Things (IIoT) Framework for Resource Management in Smart Manufacturing. IEEE Access, 8, pp.117354–117364. http://doi:10.1109/ACCESS.2020.3004711

Harsh Chaudhary; Ankit Detroja; Priteshkumar Prajapati and Parth Shah; (2020). A review of various challenges in cybersecurity using Artificial Intelligence . 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS). http://doi:10.1109/iciss49785.2020.9316003

Hossain, M. Shamim; Gobel, Stefan; Xu, Changsheng and Saddik, Abdulmotaleb El (2020). IEEE Access Special Section Editorial: Mobile Multimedia for Healthcare. IEEE Access, 8, 153799–153803. http://doi:10.1109/ACCESS.2020.3017119

Tahir, Muhammad; Sardaraz, Muhammad; Mehmood, Zahid and Muhammad, Shakoor (2020). CryptoGA: a cryptosystem based on genetic algorithm for cloud data security. Cluster Computing. http://doi:10.1007/s10586-020-03157-4

INRIA Copydays Dataset. Retrieved from https://lear.inrialpes.fr/~jegou/data.php.html

QUALINET Dataset. Retrieved from https://qualinet.github.io/databases/smart_a_light_field_image_quality_dataset/

CIFAR-10 Dataset. Retrieved from https://www.cs.toronto.edu/~kriz

/cifar.html




DOI: https://doi.org/10.31449/inf.v49i13.7514

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.