Introduction
Tympanoplasties, performed for tympanic membrane perforations, aim to protect the middle ear from external pathogens. The importance of tympanic perforation size as a prognostic factor for myringoplasty success has been debated in the literature.Reference Lee, Kelly and Mills1 However, a recent systematic review found that small perforations had a higher rate of successful tympanic membrane closure in children.Reference Hardman, Muzaffar, Nankivell and Coulson2 Therefore, accurate estimation of perforation size pre-operatively can aid patient counselling and informed consent.
Perforation size is often estimated by the clinician pre-operatively in clinic with otoscopy or otoendoscopy. This method is subject to gross errors when compared to objective photographic size.Reference Hampal, Padgham, Bunt and Wright3 Improvements in technology have seen the development of software programs which can measure such perforations with greater accuracy, but in reality they are rarely used in clinical practice.Reference Hsu, Chen, Hwang and Liu4,Reference Ibekwe, Adeosun and Nwaorgu5 One such analysis software, ImageJ, has been shown to provide a cheap and reliable method of estimating tympanic perforation size.Reference Ibekwe, Adeosun and Nwaorgu5
Our aim is to compare visual estimation versus ImageJ calculation of tympanic perforation size in the paediatric population among clinicians with different levels of experience.
Materials and methods
Five photographs of tympanic membranes were selected from a cohort of paediatric patients with tympanic membrane perforation who subsequently underwent a primary tympanoplasty at a tertiary paediatric otolaryngology unit. Patients were aged 11–16 years old. The aetiology of the perforation included a single episode of acute otitis media, history of recurrent acute otitis media and previous grommet surgery. All perforations were described as central perforations and were located anteriorly, inferiorly or in the anterior–inferior quadrant. Two patients had bilateral pathology; one patient had bilateral perforations managed surgically and one patient had contralateral squamous chronic otitis media. All patients underwent endoscopic myringoplasty. Graft material used included tragal cartilage and biodesign graft or biodesign alone. Patients with cholesteatoma were excluded. Photos were taken in the out-patient department, using a Henke Sass Wolf 0 degrees rigid sinus endoscope, and uploaded to the picture archiving and communication system.
The gold standards for comparison in our study were the perforation sizes generated by a paediatric consultant otologist using ImageJ 1.53.Reference Rasband6 This was chosen because Ibekwe et al.'s study found ImageJ to be devoid of inter-observer variation between consultant otologists therefore highlighting its reliability in this group and validating its use.Reference Ibekwe, Nwaorgu, Adeosun, Kokong, Lawal and Okundia7 The chosen images were also assessed independently by 16 other ENT clinicians who currently work in ENT departments within the region. Clinicians were either consultants, registrars or senior house officers.
Clinicians first provided a visual estimation of the size of the perforation as a percentage of the total tympanic membrane including the pars tensa and flaccida. Using ImageJ, the same clinicians traced the edges of the perforation and the tympanic membrane using a mouse or mouse pad in turn. ImageJ software calculated the area for both drawings in pixelsReference Hardman, Muzaffar, Nankivell and Coulson2 (Figure 1). The percentage perforation is calculated using the equation P/T*100 where P is the area (pixelsReference Hardman, Muzaffar, Nankivell and Coulson2) of the tympanic membrane perforation and T is the area (pixelsReference Hardman, Muzaffar, Nankivell and Coulson2) of the entire tympanic membrane including the perforation.
Figure 1. Variation from gold standard for visual estimation and for ImageJ calculation. The dark horizontal line in the box plots represents the means and the upper and lower boundaries shown by the vertical lines on either side of each box plot represent the confidence intervals. Please ignore the circles above the ImageJ box plot.
The variations between visual estimations and gold standards and ImageJ calculations and gold standards were calculated. The differences between the two methods were compared using 2-tail student's t-test. The differences between different grades were analysed using analysis of variance (ANOVA). Data were analysed using IBM® SPSS® Statistics version 21.1.
Results and analysis
The average difference from gold standard for visual estimation for all clinicians was 13.79 per cent (SD ± 6.28) and 1.62 per cent (SD ± 1.33) using ImageJ. This equates to a mean difference in deviation between methods of 12.16 per cent, 95 per cent CI (10.55, 13.78) p < 0.05 (Figure 2). On average all clinicians’ visual estimation was 2.04 times greater than the gold standard.
Figure 2. Assessment of perforation size area in pixels as generated on ImageJ. The yellow outline represents the assessment of the perforation edges on ImageJ.
The differences from gold standard for each grade are demonstrated in Table 1 and Table 2. On average, registrars were the most accurate at visually estimating perforation size, followed by consultants and senior house officers (SHOs). The difference between registrars and SHOs was statistically significant (p < 0.05) and the differences between consultants and other grades were not. In comparison, consultants performed best using ImageJ but the differences among all grades were small and not statistically significant.
Table 1. Difference between visual estimation and gold standard
SHOs = Senior House Officers
Table 2. Difference between ImageJ and gold standard
SHOs = Senior House Officers
Discussion
Our study highlights that clinicians are poor at visually estimating perforation size. Hampal et al. first compared visual estimation against an objective drawing and found greater clinical experience did not improve accuracy of estimation, with SHOs performing best, followed by registrars and finally consultants.Reference Hampal, Padgham, Bunt and Wright3 Our findings are similar to those of Hsu et al. who reported that registrars were more accurate than consultants, but they did not include SHOs in their study whilst we found no statistical significance between registrars and consultants.Reference Hsu, Chen, Hwang and Liu4 Regardless, in our study, the average difference between visual estimation and the ImageJ calculation derived by a consultant otologist was large for all groups, including registrars who were still on average inaccurate by 10 per cent.
Accurate estimation of perforation size can support parent or guardian counselling regarding conversative versus surgical management of their child's tympanic membrane perforation. Perforation size has been shown to be the single most important factor in spontaneous closure and therefore a more accurate tool of assessment is needed to aid patient counselling on the likely prognosis.Reference Jellinge, Kristensen and Larsen8 However, other patient factors such as active infection and contralateral otitis media with effusion, and surgical factors such as surgeon experience, can influence myringoplasty outcomes and should be considered.Reference Hardman, Muzaffar, Nankivell and Coulson2
ImageJ has been described as a reliable method for tympanic perforation estimation which is freely available to download.Reference Ibekwe, Adeosun and Nwaorgu5 Ibekwe et al. found it to be devoid of inter-observer variability between two experienced otologists. In clinical practice, patients with tympanic membrane perforations may be reviewed by clinicians of different grades and the reliability of ImageJ within these groups has not previously been studied.Reference Ibekwe, Adeosun and Nwaorgu5 We found that the ImageJ calculations by all grades were more accurate than visual estimation. The differences among the groups using ImageJ were small and not statistically significant, suggesting that ImageJ can be reliably used amongst clinicians of varying experience.
The accuracy of tympanic membrane perforation calculation using ImageJ depends on high-quality photos using an otoendoscope and a computer system to record and download images prior to upload onto ImageJ. One described potential limitation of an otoendoscope is the inability to focus the entire rim of the tympanic membrane in one single image.Reference Ibekwe, Adeosun and Nwaorgu5 A previous study using cadaveric temporal bone models widened the external auditory canal to ensure the tympanic membranes were photographed occupying only the central three-quarters of the field of view.Reference Hampal, Padgham, Bunt and Wright3 Young children are less compliant with examination and multiple attempts to capture high-quality, in-focus images may be futile. In addition, uploading and tracing the image on ImageJ is time consuming and potentially unfeasible within a patient's allocated clinic slot.
Otoendoscopy provides additional benefits for the clinical consultation. Image capture allows clinicians to compare their current findings to previous examinations particularly if they have not assessed the patient previously themselves. Furthermore, traditional examination with an otoscope or microscope excludes the patient or carer from observing the examination. Otoendoscopy, however, allows the clinician to share their findings and therefore engage the user.Reference Wong, Schwam, Arrighi-Allisan, Fan, Perez and Cosetti9 Wong et al. found that adults reported greater satisfaction regarding communication and technical quality when video otoscopy was utilised in an out-patient clinic compared to standard microscopy.Reference Wong, Schwam, Arrighi-Allisan, Fan, Perez and Cosetti9 In the paediatric population, improved parental satisfaction and patient centeredness with video-endoscopy was found with children examined in the emergency department.Reference Rimon, Avraham, Sharabi-Nov, Luder, Krupik and Gilbey10 The use of video-otoscopy or video capture at a high rate per frame with direct upload to ImageJ may negate some limiting factors to support its introduction into daily clinical practice.
Our study is limited by the small number of images of similar perforation sizes in similar positions which do not reflect the full spectrum of perforations clinicians will encounter. Whilst variation in perforation size may affect the accuracy of visual estimation, we do not expect it to affect the ImageJ calculation and therefore still consider ImageJ a reliable tool for assessment. A paediatric consultant otologist's ImageJ results were selected as the gold standard which can be subject to error. The authors justify the choice of gold standard, given the reliability of results by consultant otologists demonstrated by Ibekwe et al.Reference Ibekwe, Nwaorgu, Adeosun, Kokong, Lawal and Okundia7 In their study, two consultant otologists reviewed 100 tympanic membranes using ImageJ software and conventional direct vision otoscopy and found the latter to be devoid of inter-user error.
• Tympanic membrane perforation size is an important prognostic factor for myringoplasty success
• Estimation of perforation size using otoscopy is subject to error
• Use of ImageJ software for analysing otoendoscopic images has been shown to be a reliable method of calculating perforation size by consultant otologists
• Using ImageJ software is reliable when used by ENT clinicians of all grades and not just consultant otologists
Conclusion
ImageJ is a reliable method for assessment of tympanic membrane perforation size and can be used by all ENT clinicians regardless of experience. However, we highlight practical limitations that should be addressed to facilitate its use in daily clinical practice.
Funding
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
None declared.
