Co-Mask R-CNN: collaborative learning-based method for tooth instance segmentation

Traditional tooth image analysis methods primarily focus on feature extraction from individual images, often overlooking critical tooth shape and position information. This paper presents a novel computer-aided diagnosis method, Collaborative learning with Mask Region-based Convolutional Neural Network (Co-Mask R-CNN), designed to enhance tooth image analysis by leveraging the integration of complementary information. First, image enhancement is employed to generate an edge-enhanced tooth edge image. Then, a collaborative learning strategy combined with Mask R-CNN is introduced, where the original and edge images are input simultaneously, and a two-stream encoder extracts feature maps from complementary images. By utilizing an attention mechanism, the output features from the two branches are dynamically fused, quantifying the relative importance of the two complementary images at different spatial positions. Finally, the fused feature map is utilized for tooth instance segmentation. Extensive experiments are conducted using a proprietary dataset to evaluate the effectiveness of Co-Mask R-CNN, and the results are compared against those of an alternative segmentation network. The results demonstrate that Co-Mask R-CNN outperforms the other networks in terms of both segmentation accuracy and robustness. Consequently, this method holds considerable promise for providing medical professionals with precise tooth segmentation results, establishing a reliable foundation for subsequent tooth disease diagnosis and treatment.

Deep learning; Two-stream collaborative network; Tooth instance segmentation; Bitewing radiograph

[1] Kumar R, Saha P. A review on artificial intelligence and machine learning to improve cancer management and drug discovery. International Journal for Research in Applied Sciences and Biotechnology. 2022; 9: 149–156.

[2] Huang C, Wang J, Wang S, Zhang Y. A review of deep learning in dentistry. Neurocomputing. 2023; 554: 126629.

[3] Helli S, Hamamci A. Tooth instance segmentation on panoramic dental radiographs using u-nets and morphological processing. Duzce University Journal of Science and Technology. 2022; 10: 39–50.

[4] Martins M V, Baptista L, Luís H, Assunção V, Araújo MR, Realinho V. Machine learning in x-ray diagnosis for oral health: a review of recent progress. Computation. 2023; 11: 115.

[5] Zhang J, Jiang Y, Gao F, Zhao S, Yang F, Song L. A fast automatic reconstruction method for panoramic images based on cone beam computed tomography. Electronics. 2022; 11: 2404.

[6] Polizzi A, Quinzi V, Ronsivalle V, Venezia P, Santonocito S, Giudice A Lo, et al. Tooth automatic segmentation from CBCT images: a systematic review. Clinical Oral Investigations. 2023: 27: 3363–3378.

[7] Bruellmann D, Sander S, Schmidtmann I. The design of an fast fourier filter for enhancing diagnostically relevant structures—endodontic files. Computers in Biology and Medicine. 2016; 72: 212–217.

[8] Oltu B, Karaca BK, Uyar T, Uyar DS. Detection of occlusal plaque and caries using fuzzy C means based segmentation algorithm. 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA) (pp. 1–5). IEEE. 2021.

[9] Wang C, Huang C, Lee J, Li C, Chang S, Siao M, et al. A benchmark for comparison of dental radiography analysis algorithms. Medical Image Analysis. 2016; 31: 63–76.

[10] Patil S, Kulkarni V, Bhise A. Algorithmic analysis for dental caries detection using an adaptive neural network architecture. Heliyon. 2019; 5: e01579.

[11] Yau H, Yang T, Chen Y. Tooth model reconstruction based upon data fusion for orthodontic treatment simulation. Computers in Biology and Medicine. 2014; 48: 8–16.

[12] Wang Y, Liu S, Wang G, Liu Y. Accurate tooth segmentation with improved hybrid active contour model. Physics in Medicine and Biology. 2018; 64: 015012.

[13] Salimzadeh S, Kandulu S. Teeth segmentation of bitewing X-ray images using wavelet transform. Informatica. 2020; 44: 421–426.

[14] Modi CK, Desai NP. A simple and novel algorithm for automatic selection of ROI for dental radiograph segmentation. 2011 24th Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 000504–000507). IEEE. 2011.

[15] He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. 2017 IEEE International Conference on Computer Vision. 2017; 2961–2969.

[16] Jader G, Fontineli J, Ruiz M, Abdalla K, Pithon M, Oliveira L. Deep instance segmentation of teeth in panoramic X-ray images. 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). 2018; 400–407.

[17] Pinheiro L, Silva B, Sobrinho B, Lima F, Cury P, Oliveira L. Numbering permanent and deciduous teeth via deep instance segmentation in panoramic X-rays. 17th International Symposium on Medical Information Processing and Analysis. 2021; 12088: 95–104.

[18] AlQarni S, Chandrashekar G, Bumann EE, Lee Y. Incremental learning for panoramic radiograph segmentation. 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2022; 557–561.

[19] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems. 2015; 28.

[20] Lee J, Han S, Kim YH, Lee C, Kim I. Application of a fully deep convolutional neural network to the automation of tooth segmentation on panoramic radiographs. Oral Surgery, Oral Medicine, Oral Pathology, and Oral Radiology. 2020; 129: 635–642.

[21] Zhao Y, Li P, Gao C, Liu Y, Chen Q, Yang F, et al. TSASNet: tooth segmentation on dental panoramic X-ray images by two-stage attention segmentation network. Knowledge-Based Systems. 2020; 206: 106338.

[22] Chung M, Lee M, Hong J, Park S, Lee J, Lee J, et al. Pose-aware instance segmentation framework from cone beam CT images for tooth segmentation. Computers in Biology and Medicine. 2020; 120: 103720.

[23] Silva B, Pinheiro L, Oliveira L, Pithon M. A study on tooth segmentation and numbering using end-to-end deep neural networks. 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). 2020; 164–171.

[24] Wang K, Liew JH, Zou Y, Zhou D, Feng J. PANet: few-shot image semantic segmentation with prototype alignment. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019; 9197–9206.

[25] Zhang H, Wu C, Zhang Z, Zhu Y, Lin H, Zhang Z, et al. ResNeSt: split-attention networks. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2022; 2736–2746.

[26] Chen K, Pang J, Wang J, Xiong Y, Li X, Sun S, et al. Hybrid task cascade for instance segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019; 4974–4983.

[27] Leite AF, Gerven AV, Willems H, Beznik T, Lahoud P, Gaêta-Araujo H, et al. Artificial intelligence-driven novel tool for tooth detection and segmentation on panoramic radiographs. Clinical Oral Investigations. 2021; 25: 2257–2267.

[28] Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, et al. Dual attention network for scene segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019; 3146–3154.

[29] Zhou Y, He X, Huang L, Liu L, Zhu F, Cui S, et al. Collaborative learning of semi-supervised segmentation and classification for medical images. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019; 2079–2088.

[30] Kumar A, Fulham M, Feng D, Kim J. Co-learning feature fusion maps from pet-CT images of lung cancer. IEEE Transactions on Medical Imaging. 2020; 39: 204–217.

[31] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016; 770–778.

[32] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C, et al. SSD: single shot multibox detector. Computer Vision—ECCV 2016. 2016; 104: 21–37.

[33] Redmon J, Farhadi A. Yolov3: An incremental improvement[J]. arXiv preprint arXiv:1804.02767, 2018.

[34] Wang C, Bochkovskiy A, Liao HM. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2023; 7464–7475.