Jian Chen,1,2,* Hongwei Wang,1,2,* Zihao Zhang,3 Kaijian Xia,1,2 Fuli Gao,1 Xiaodan Xu,1 Ganhong Wang4 

1Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Suzhou, Jiangsu, 215500, People’s Republic of China; 2Center of Intelligent Medical Technology Research, Changshu Hospital Affiliated to Soochow University, Suzhou, 215500, People’s Republic of China; 3Department of Information Engineering, Shanghai Haoxiong Education Technology Co., Ltd, Shanghai, 200434, People’s Republic of China; 4Department of Gastroenterology, Changshu Hospital Affiliated to Nanjing University of Chinese Medicine, Suzhou, Jiangsu, 215500, People’s Republic of China

*These authors have contributed equally to this work

Correspondence: Ganhong Wang; Xiaodan Xu, Email 651943259@qq.com; xxddocter@gmail.com

Objective: To develop a multi-task artificial intelligence-assisted system for small bowel capsule endoscopy (SBCE) based on various Transformer neural network architectures. The system integrates lesion recognition, cumulative time statistics, and progress bar marking functions to enhance the efficiency and accuracy of endoscopic image interpretation while effectively reducing missed diagnoses.
Methods: A dataset comprising 12 annotated categories of images captured by three different brands of capsule endoscopy devices was collected. Transfer learning and fine-tuning were conducted on five pre-trained Transformer models. Performance metrics, including accuracy, sensitivity, specificity, and recognition speed, were evaluated to select the best-performing model. The optimal model was converted from PyTorch to Open Neural Network Exchange (ONNX) format. Using OpenCV and MMCV tools, a multi-task SBCE-assisted reading system was developed.
Results: A total of 34,799 images were included in the study. The best-performing model, FocalNet, achieved a weighted average sensitivity of 85.69%, specificity of 98.58%, accuracy of 85.69%, and an AUC of 0.98 across all categories. Its diagnostic accuracy outperformed junior physicians (χ²=17.26, p< 0.05) and showed no statistical difference compared to senior physicians (χ²=0.0716, p> 0.05). The multi-task AI-assisted reading system, “FocalCE-Master”, developed based on FocalNet, achieved a diagnostic speed of 592.40 frames per second, significantly faster than endoscopists. By integrating cumulative time bar charts with progress bar marking functionality, the system enables rapid localization and review of lesions, effectively streamlining the diagnostic workflow of SBCE.
Conclusion: The multi-task SBCE-assisted reading system developed using Transformer networks demonstrated rapid and accurate classification of various small bowel lesions. It holds significant potential in enhancing diagnostic efficiency and image review speed for endoscopists. However, the AI system has not yet been validated in prospective clinical trials, and further real-world studies are needed to confirm its clinical applicability.

Keywords: small bowel lesions, artificial intelligence, small bowel capsule endoscopy, transformer, transfer learning