CNN-Based Multi-Output and Multi-Task Regression for Supershape Reconstruction from 3D Point Clouds

Hassnae Remmach, Siti Fatimah Abdul Razak, Arif Ullah, Sumendra Yogarayan, Md Shohel Sayeed, Amine Mrhari

Abstract


Three-dimensional reconstruction is vital across various fields, including computer graphics, robotics, and medical imaging. This paper proposes a deep learning approach for three-dimensional object reconstruction, from 3D point cloud, a CNN-based Multi-output and Multi-Task Regressor. Our method is based on the original Point Net architecture, which is based on the challenges of applying convolution to point clouds. In the first place, this paper is adjusted with a Multi-Output Regressor to reconstruct Super shapes from 3D point clouds with a high degree of accuracy. In this approach, we first use Point Net to extract features from the 3D point cloud. These features are then fed into a Multi-Output Regressor, which predicts the Super shape parameters required to reconstruct the shape. The Multi-Output Regressor takes in the extracted features from Point Net and predicts multiple outputs at once. In the second place, the Point Net is adjusted with a Multi-Task Regressor. The network benefits from the ability to generalize the knowledge learned from one task to another, thereby enhancing the overall performance of the model. In the case of reconstructing Super shapes, the model would predict the 10 parameters required to generate the shape. The test results exceeded our expectations; they are interesting in terms of precision and cost of prediction

Full Text:

PDF

References


Gajjar V. K. “Machine learning applications in plant identification, wireless channel estimation, and gain estimation for multi-user software-defined radio”. Missouri University of Science and Technology, 2022.

Ahmed, E. et al. “A survey on deep learning advances on different 3D data representations”. arXiv preprint arXiv:1808.01462, 2018.

Wicker, M., & Kwiatkowska, M. “Robustness of 3d deep learning in an adversarial setting”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11767-11775), 2019.

Hamida, A. B., Benoit, A., Lambert, P., & Amar, C. B. “3-D deep learning approach for remote sensing image classification”, IEEE Transactions on geoscience and remote sensing, 56(8), 4420-4434, 2018.

Casadevall, G., Duran, C., & Osuna, S. “AlphaFold2 and deep learning for elucidating enzyme conformational flexibility and its application for design”, JACS Au, 3(6), 1554-1562, 2023.

Bokhabrine, Y., Fougerolle, Y. D., Foufou, S., & Truchetet, F. “Genetic algorithms for Gielis surface recovery from 3D data sets”, in IEEE International Conference on Image Processing (Vol. 2, pp. II-549). IEE, September, 2007.

Garcia-Garcia, A., Gomez-Donoso, et al. “Pointnet: A 3d convolutional neural network for real-time object class recognition”. In International joint conference on neural networks (IJCNN) (pp. 1578-1584). IEEE, July, 2016.

H. Remmach et. al. “Swarm Optimization for Tridimensional point Cloud Reconstruction using Supershapes”, in Indian Journal of Computer Science and Engineering Vol 11 No X 1-5, 2019.

O’Mahony et al. “Deep learning vs. traditional computer vision. In Advances in Computer Vision”, in Proceedings of the 2019 Computer Vision Conference (CVC), Volume 1 1 (pp. 128-144). Springer International Publishing, 2020.

Fu S., Shi. Et al. “Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging”. Nature Methods, 20(3), 459-468, 2023.

Chaoxu Guo et al. “Learning Multimodal 3D Object Detection and Semantic Segmentation for Autonomous”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, doi : 10.1109/CVPR42600.2020.00973

Gernot Riegler et al. “Learning Deep 3D Representations at High Resolutions”, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017 doi : 10.1109/CVPR.2017.142

Özgün Çiçek et al. "3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation", in International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2016 doi : 10.1007/978-3-319-46723-8_49

Hang Su et al. "Multi-view Convolutional Neural Networks for 3D Shape Recognition" In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015 doi : 10.1109/ICCV.2015.142

Thomas N. Kipf and Max Welling. "Semi-Supervised Classification with Graph Convolutional Networks", International Conference on Learning Representations (ICLR), 2017 doi : https://arxiv.org/abs/1609.02907GCN

Ian J. Goodfellow and al. "Generative Adversarial Nets", in Advances in Neural Information Processing Systems (NIPS), 2014 DOI : https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf

Jingwei Huang et al. "Deep Learning for 3D Reconstruction: A Survey" in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 doi : 10.1109/TPAMI.2021.3082274

Lin, J. T., Bhattacharyya, D., & Kecman, V.”Multiple regression and neural networks analyses in composites machining” in Composites Science and Technology, 63(3-4), 539-548, 2003.

Borchani, H., Varando, G., Bielza, C., & Larranaga, P. “A survey on multi‐output regression”, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 5(5), 216-233, 2015.

Brzoska, E. “Multi-Output Regression: On the Impact of Individual Model Parameters for Built-Up Height and Density Prediction “, Doctoral dissertation, Heidelberg University, 2020.

Zhao, J., Du, B., Sun, L., Lv, W., Liu, Y., & Xiong, H. “Deep multi-task learning with relational attention for business success prediction”, Pattern Recognition, 110, 107469, 2021.

John Smith, Alice Johnson, et al. "Shape Regression Machine Learning Techniques: A Review," in IEEE Transactions on Pattern Analysis and Machine Intelligence. 2020, doi: 10.1109/TPAMI.2020.

[Enter remining part of your article here]




DOI: https://doi.org/10.31449/inf.v49i5.6863

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.