Hostname: page-component-586b7cd67f-tf8b9 Total loading time: 0 Render date: 2024-11-24T20:05:54.595Z Has data issue: false hasContentIssue false
  • English
  • Français

Fast Noninvasive Morphometric Characterization of Free Human Sperms Using Deep Learning

Published online by Cambridge University Press:  24 June 2022

Guole Liu ,
Hao Shi ,
Huan Zhang ,
Yating Zhou ,
Yujiao Sun ,
Wei Li ,
Xuefeng Huang ,
Yuqiang Jiang ,
Yaliang Fang  and
Ge Yang Open the ORCID record for Ge Yang [Opens in a new window]
Guole Liu
Affiliation:
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Hao Shi
Affiliation:
Sperm Capturer (Beijing) Biotechnology Co. Ltd., Beijing 100070, China
Huan Zhang
Affiliation:
Reproductive Medicine Center, The First Affiliated Hospital of Wenzhou Medical University, Zhejiang 325000, China
Yating Zhou
Affiliation:
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Yujiao Sun
Affiliation:
State Key Laboratory of Molecular Developmental Biology, Institute of Genetics and Developmental Biology, Chinese Academy of Sciences, Beijing 100101, China
Wei Li
Affiliation:
Beijing Children's Hospital, Capital Medical University, Beijing 100045, China
Xuefeng Huang
Affiliation:
Reproductive Medicine Center, The First Affiliated Hospital of Wenzhou Medical University, Zhejiang 325000, China
Yuqiang Jiang
Affiliation:
State Key Laboratory of Molecular Developmental Biology, Institute of Genetics and Developmental Biology, Chinese Academy of Sciences, Beijing 100101, China
Yaliang Fang*
Affiliation:
Sperm Capturer (Beijing) Biotechnology Co. Ltd., Beijing 100070, China
Ge Yang*
Affiliation:
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
*
*Corresponding author: Ge Yang, Email: [email protected]; Yaliang Fang, Email: [email protected]
*Corresponding author: Ge Yang, Email: [email protected]; Yaliang Fang, Email: [email protected]
Get access

Abstract

The selection of high-quality sperms is critical to intracytoplasmic sperm injection, which accounts for 70–80% of in vitro fertilization (IVF) treatments. So far, sperm screening is usually performed manually by clinicians. However, the performance of manual screening is limited in its objectivity, consistency, and efficiency. To overcome these limitations, we have developed a fast and noninvasive three-stage method to characterize morphology of freely swimming human sperms in bright-field microscopy images using deep learning models. Specifically, we use an object detection model to identify sperm heads, a classification model to select in-focus images, and a segmentation model to extract geometry of sperm heads and vacuoles. The models achieve an F1-score of 0.951 in sperm head detection, a z-position estimation error within ±1.5 μm in in-focus image selection, and a Dice score of 0.948 in sperm head segmentation, respectively. Customized lightweight architectures are used for the models to achieve real-time analysis of 200 frames per second. Comprehensive morphological parameters are calculated from sperm head geometry extracted by image segmentation. Overall, our method provides a reliable and efficient tool to assist clinicians in selecting high-quality sperms for successful IVF. It also demonstrates the effectiveness of deep learning in real-time analysis of live bright-field microscopy images.

Keywords

bright-field microscopydeep learninghuman spermintracytoplasmic sperm injectionsperm morphology
Type
Biological Applications
Information
Microscopy and Microanalysis , Volume 28 , Issue 5 , October 2022 , pp. 1767 - 1779
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press on behalf of the Microscopy Society of America

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agarwal, A, Mulgund, A, Hamada, A & Chyatte, MR (2015). A unique view on male infertility around the globe. Reprod Biol Endocrinol 13(1), 1–9.CrossRefGoogle ScholarPubMed
Bijar, A, Mikaeili, M, Benavent, AP & Khayati, R (2012). Segmentation of sperm's acrosome, nucleus and mid-piece in microscopic images of stained human semen smear. In 8th International Symposium on Communication Systems, Networks & Digital Signal Processing, pp. 1–6.Google Scholar
Boitrelle, F, Guthauser, B, Alter, L, Bailly, M, Wainer, R, Vialard, F, Albert, M & Selva, J (2013). The nature of human sperm head vacuoles: A systematic literature review. Basic Clin Androl 23(1), 1–9.CrossRefGoogle ScholarPubMed
Brenner, JF, Dew, BS, Horton, JB, King, T, Neurath, PW & Selles, WD (1976). An automated microscope for cytologic research a preliminary evaluation. J Histochem Cytochem 24(1), 100–111.CrossRefGoogle ScholarPubMed
Carrillo, H, Villarreal, J, Sotaquira, M, Goelkel, A & Gutierrez, R (2007). A computer aided tool for the assessment of human sperm morphology. In IEEE 7th International Symposium on BioInformatics and BioEngineering, pp. 1152–1157.CrossRefGoogle Scholar
Chen, L-C, Papandreou, G, Schroff, F & Adam, H (2017). Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587.Google Scholar
Cui, W (2010). Mother or nothing: The agony of infertility. Bull World Health Organ 88(12), 881–882.CrossRefGoogle ScholarPubMed
Dai, C, Zhang, Z, Huang, J, Wang, X, Ru, C, Pu, H, Xie, S, Zhang, J, Moskovtsev, S, Librach, C, Jarvi, K & Sun, Y (2018). Automated non-invasive measurement of single sperm's motility and morphology. IEEE Trans Med Imaging 37(10), 2257–2265.CrossRefGoogle ScholarPubMed
Dai, J, Li, Y, He, K & Sun, J (2016). R-FCN: Object detection via region-based fully convolutional networks. In Advances in Neural Information Processing Systems (NeurIPS), pp. 379–387.Google Scholar
Dardikman-Yoffe, G, Mirsky, SK, Barnea, I & Shaked, NT (2020). High-resolution 4-D acquisition of freely swimming human sperm cells without staining. Sci Adv 6(15), eaay7619.CrossRefGoogle ScholarPubMed
De, K & Masilamani, V (2013). Image sharpness measure for blurred images in frequency domain. Procedia Eng 64, 149–158.CrossRefGoogle Scholar
Donnelly, ET, Lewis, SE, McNally, JA & Thompson, W (1998). In vitro fertilization and pregnancy rates: The influence of sperm motility and morphology on IVF outcome. Fertil Steril 70(2), 305–314.CrossRefGoogle ScholarPubMed
Enginsu, M, Dumoulin, J, Pieters, M, Bras, M, Evers, J & Geraedts, J (1991). Evaluation of human sperm morphology using strict criteria after Diff-Quik staining: Correlation of morphology with fertilization in vitro. Hum Reprod 6(6), 854–858.CrossRefGoogle ScholarPubMed
Farhadi, A & Redmon, J (2018). YOLOv3: An incremental improvement. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1804–2767.Google Scholar
Gander, W, Golub, GH & Strebel, R (1994). Least-squares fitting of circles and ellipses. BIT Numer Math 34(4), 558–578.CrossRefGoogle Scholar
Ghasemian, F, Mirroshandel, SA, Monji-Azad, S, Azarnia, M & Zahiri, Z (2015). An efficient method for automatic morphological abnormality detection from human sperm images. Comput Methods Programs Biomed 122(3), 409–420.CrossRefGoogle ScholarPubMed
Graves, A (2013). Generating sequences with recurrent neural networks. arXiv preprint, arXiv:1308.0850.Google Scholar
Guzick, DS, Overstreet, JW, Factor-Litvak, P, Brazil, CK, Nakajima, ST, Coutifaris, C, Carson, SA, Cisneros, P, Steinkampf, MP & Hill, JA (2001). Sperm morphology, motility, and concentration in fertile and infertile men. N Engl J Med 345(19), 1388–1393.CrossRefGoogle ScholarPubMed
He, K, Zhang, X, Ren, S & Sun, J (2016). Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778.CrossRefGoogle Scholar
Ilhan, H, Serbes, G & Aydin, N (2019). Automatic directional masking technique for better sperm morphology segmentation and classification analysis. Electron Lett 55(5), 256–258.CrossRefGoogle Scholar
Javadi, S & Mirroshandel, SA (2019). A novel deep learning method for automatic assessment of human sperm images. Comput Biol Med 109, 182–194.CrossRefGoogle ScholarPubMed
Kingma, DP & Ba, J (2014). Adam: A method for stochastic optimization. arXiv preprint, arXiv:1412.6980.Google Scholar
Krizhevsky, A, Sutskever, I & Hinton, GE (2012). Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NeurIPS), pp. 1097–1105.Google Scholar
Lin, T-Y, Goyal, P, Girshick, R, He, K & Dollár, P (2017). Focal loss for dense object detection. In IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988.CrossRefGoogle Scholar
Liu, W, Anguelov, D, Erhan, D, Szegedy, C, Reed, S, Fu, C-Y & Berg, AC (2016). SSD: Single shot multibox detector. In European Conference on Computer Vision (ECCV), pp. 21–37.CrossRefGoogle Scholar
Long, J, Shelhamer, E & Darrell, T (2015). Fully convolutional networks for semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440.CrossRefGoogle Scholar
Mirroshandel, SA & Ghasemian, F (2018). Automated morphology detection from human sperm images. In Intracytoplasmic Sperm Injection: Indications, Techniques and Applications, Palermo, GD & Sills, ES (Eds.), pp. 99–122. Cham: Springer International Publishing.CrossRefGoogle Scholar
Movahed, RA & Orooji, M (2018). A learning-based framework for the automatic segmentation of human sperm head, acrosome and nucleus. In 25th National and 3rd International Iranian Conference on Biomedical Engineering, pp. 1–6.CrossRefGoogle Scholar
Papageorgiou, CP, Oren, M & Poggio, T (1998). A general framework for object detection. In IEEE International Conference on Computer Vision (ICCV), pp. 555–562.CrossRefGoogle Scholar
Pertuz, S, Puig, D & Garcia, MA (2013). Analysis of focus measure operators for shape-from-focus. Pattern Recognit 46(5), 1415–1432.Google Scholar
Redmon, J, Divvala, S, Girshick, R & Farhadi, A (2016). You only look once: Unified, real-time object detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788.CrossRefGoogle Scholar
Ren, S, He, K, Girshick, R & Sun, J (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems (NeurIPS), pp. 91–99.Google Scholar
Ronneberger, O, Fischer, P & Brox, T (2015). U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 234–241.Google Scholar
Rubino, P, Viganò, P, Luddi, A & Piomboni, P (2015). The ICSI procedure from past to future: A systematic review of the more controversial aspects. Hum Reprod Update 22(2), 194–227.Google ScholarPubMed
Schmidhuber, J (2015). Deep learning in neural networks: An overview. Neural Netw 61, 85–117.CrossRefGoogle ScholarPubMed
Setti, AS, Ferreira, RC, Braga, D, Figueira, R, Iaconelli, A Jr & Borges, E Jr (2010). Intracytoplasmic sperm injection outcome versus intracytoplasmic morphologically selected sperm injection outcome: A meta-analysis. Reprod Biomed Online 21(4), 450–455.CrossRefGoogle Scholar
Shaker, F, Monadjemi, SA & Naghsh-Nilchi, AR (2016). Automatic detection and segmentation of sperm head, acrosome and nucleus in microscopic images of human semen smears. Comput Methods Programs Biomed 132, 11–20.CrossRefGoogle ScholarPubMed
Shapiro, LG (1996). Connected component labeling and adjacency graph construction. In Machine Intelligence and Pattern Recognition, Kong, TY & Rosenfeld, A (Eds.), pp. 1–30. North-Holland: Elsevier.Google Scholar
Shi, J & Malik, J (2000). Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8), 888–905.Google Scholar
Simonyan, K & Zisserman, A (2015). Very deep convolutional networks for large-scale image recognition. In Computing Research Repository (CoRR), abs/1409.1556.Google Scholar
Snell, J, Ridgeway, K, Liao, R, Roads, BD, Mozer, MC & Zemel, RS (2017). Learning to generate images with perceptual similarity metrics. In IEEE International Conference on Image Processing (ICIP), pp. 4277–4281.CrossRefGoogle Scholar
Szegedy, C, Liu, W, Jia, Y, Sermanet, P, Reed, S, Anguelov, D, Erhan, D, Vanhoucke, V & Rabinovich, A (2015). Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9.CrossRefGoogle Scholar
Vanderzwalmen, P, Hiemer, A, Rubner, P, Bach, M, Neyer, A, Stecher, A, Uher, P, Zintz, M, Lejeune, B & Vanderzwalmen, S (2008). Blastocyst development after sperm selection at high magnification is associated with size and number of nuclear vacuoles. Reprod Biomed Online 17(5), 617–627.CrossRefGoogle ScholarPubMed
Viola, P & Jones, M (2001). Robust real-time object detection. Int J Comput Vis 4(4), 34–47.Google Scholar
Zhou, Z, Siddiquee, MMR, Tajbakhsh, N & Liang, J (2018). Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 3–11. Springer.CrossRefGoogle Scholar
Supplementary material: PDF

Liu et al. supplementary material

Liu et al. supplementary material 1

Download Liu et al. supplementary material(PDF)
PDF 262.1 KB

Liu et al. supplementary material

Liu et al. supplementary material 2

Download Liu et al. supplementary material(Audio)
Audio 2.3 MB