Over the past two decades, the use of residency websites has steadily increased. Websites serve as a major resource for applicants to learn about different programs and shape their decision to apply. According to surveys conducted among applicants in the 2020–2021 residency match process for Canadian neurology and American plastic surgery programs, program websites were used by 87% of applicants. Reference Niznick, Lun and Gotfrit1,Reference Pflibsen, Deckey, Brinkman, Tummala, Casey and Teven2
Studies of residency program website comprehensiveness and utility have shown applicants valuing website content such as faculty and resident profiles, research opportunities, and curriculum descriptions. Reference Chen, Hoang and Garner3 While gaps in the website content of American neurology programs have been identified in the pre-pandemic era, Reference Daniel, Vila, Leon Guerrero and Karroum4 there have been no investigations regarding how program websites have been affected by COVID-19 nor how American and Canadian program websites differ. With limited access to visiting electives due to COVID-19, program websites are expected to play a more significant role in applicant decision-making.
While neurology residency programs have increased social media presence as a response to the pandemic, it is unclear how websites have evolved to adapt. Reference Gaini, Patel, Khan, Singh and Love5 We evaluated the quality and comprehensiveness of neurology residency program websites across North America in the setting of COVID-19 and compared American and Canadian program websites to provide insight into how programs can optimize website content to tailor to applicant needs.
A list of adult neurology programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) in the continental United States (US) and by the Royal College of Physicians and Surgeons in Canada was generated through the Fellowship and Residency Electronic Interactive Database and Canadian Resident Matching Service (CaRMS) website, respectively. Data on the number of postgraduate year 1 (PGY1) positions offered in 2022 were collected through Residency Explorer and CaRMS for American and Canadian programs, respectively. The 2022 US News & World Report’s best neurology and neurosurgery hospitals list was used to rank US programs. Due to the absence of a ranking system for Canadian neurology divisions/departments, program rank, and website were not compared for Canada. Two reviewers (CT, WW) extracted data independently and in duplicate from websites in August 2022. Any discrepancies were resolved through consensus and discussion with a senior author (FK).
Adapted from validated studies, an assessment tool for components required in an extensive program website was created. Reference Jain, Misra and Sood6,Reference Khan, Salman and Yasmin7 A 40-item point system was used to assess residency program websites consisting of the following domains: recruitment, faculty, residents, education and research, teaching, benefits, and community and environment (Table 1). A point was assigned for each criterion fulfilled either on the official program website or on webpages and documents linked to the official website. Details for criteria fulfillment and its application are summarized in the Supplementary methods.
Table 1: Evaluation criteria and proportion fulfilled by adult neurology residency program websites in the United States (US) (N = 171 a ) and Canada (N = 16)
a Three of 174 US neurology residency programs had inaccessible websites.
To assess regional differences, US and Canadian programs were stratified based on the geographical subdivision system used by the National Geographic Society and Government of Canada, respectively. The US was grouped into five regions: Midwest, Northeast, Southeast, Southwest, and West. Canada was stratified into four regions: Atlantic, Central, Prairies, and West.
Data for fulfilled criteria were summarized by mean and standard deviation of scores, stratified by criteria domain and country. Distribution of program websites by degree of comprehensiveness is summarized in Figure 1. For the US, national program rank and number of PGY1 positions were evaluated as singular predictors of total website score using simple linear regression. If significant, predictors were evaluated using multivariable regression to assess collinearity. The predictiveness of domain sub-scores for program rank was assessed by multivariable linear regression. For Canada, the number of PGY1 seats and total website score were assessed using simple linear regression. Regional subgroup analysis for total website score evaluation was conducted with the Kruskal-Wallis test with post hoc pairwise comparison using false detection rate for continuity correction. Pairwise comparison of total score and domain sub-scores between American and Canadian programs was performed with the Mann-Whitney U test. p < 0.05 was considered statistically significant. Analyses were completed in RStudio 2022.02.0 + 443.
Figure 1: Distribution of US and Canadian neurology residency program websites by degree of comprehensiveness. The degree of website comprehensiveness is based on the number of evaluation criteria fulfilled (refer to Table 1 for the criteria list). Extensive = 28–40, moderate = 14–27, limited = 0–13 criteria fulfilled. Of US neurology residency program websites, 75, 88, and 8 websites were comprehensive to an extensive, moderate, and limited degree, respectively. Of Canadian neurology residency program websites, 1, 13, and 2 websites were comprehensive to an extensive, moderate, and limited degree, respectively.
We identified 174 US and 16 Canadian adult neurology residency programs. Three of 174 ACGME-accredited neurology programs had inaccessible websites and were excluded. Proportion of fulfilled criteria and website evaluation scores are summarized in Tables 1 and 2, respectively. US programs had higher website scores than Canadian programs in total score (p = 0.002) and the domains of faculty (p = 0.003), residents (p < 0.0001), education and research (p = 0.0005), benefits (p < 0.0001), and community and environment (p < 0.0001). Canadian programs had higher website scores in the teaching domain than US programs (p = 0.002).
Among US programs, total website score was positively associated with program ranking (p = 0.002). The domain sub-score for education and research was positively associated with program rank (p = 0.0002), when adjusted for other domain contributions. Among all North American programs, there was an inverse association between total website score and number of PGY1 seats (p < 0.0001). This difference remained significant only in the US context, even when adjusted for program ranking (p = 0.04). In regional analysis, Canadian regions did not exhibit any difference in domain sub-scores (p > 0.05). Within the US, the Southwest had higher faculty domain sub-scores compared to the Midwest, Southeast, and West individually (all p = 0.007). The US Midwest had higher education and research domain sub-scores than the Southwest (p = 0.02).
Our study reveals variability in content extensiveness between American and Canadian neurology residency program websites. While Canadian program websites had more comprehensive content in teaching, US program websites were more extensive in faculty, resident, education and research, benefits, and community and environment content. In US programs, higher website total scores and education and research domain sub-scores were associated with better rankings. Among US programs, higher website scores correlated with fewer PGY1 seats. Across US regions, the Southwest had higher faculty domain sub-scores than the Midwest, Southeast, and West, while the Midwest had higher education and research sub-scores than the Southwest.
Prior studies have shown that applicants value faculty, resident, research, and curriculum information across specialties. Reference Khan, Salman and Yasmin7,Reference Gaeta, Birkhahn, Lamont, Banga and Bove8 This may explain why we found that websites provided more comprehensive information in these domains. As in-person electives were suspended during COVID-19, websites may have been curated to inform more about community, environment, and benefits, that would have been otherwise learned in-person. Further, differences between Canadian and US program websites may be attributed to differences in program values. For instance, all Canadian neurology residency programs have academic half-days, highlighting the importance of resident teaching in Canada. Reference Chalk9 Higher education sub-scores of Canadian programs (all university-based) compared to those of the US (70% university-based) Reference Daniel, Vila, Leon Guerrero and Karroum4 are consistent with the proposal that university-based programs have higher quality curriculum than community-based programs. Reference Chen, Saidi, Rivkees and Black10 However, further study is warranted.
Intriguingly, we found associations of website comprehensiveness with better program ranking and fewer PGY1 seats in the US. Comprehensive websites can foster trust and confidence in programs, attract larger and diverse applicant pools, and increase program visibility and recognition, all of which can lead to better rankings. Notably, US News & World Report’s rankings largely rely on patient outcomes, independent of website content. 11 Furthermore, programs with fewer PGY1 seats may invest more in website content due to greater resource availability or emphasis placed on attracting qualified applicants. Regional differences in program websites may be attributed to elements such as demographics, culture, resource availability, and local competition. Ultimately, further investigation in these potentially bidirectional associations is needed.
Previously, a pre-pandemic study evaluating US neurology residency program websites found associations between comprehensiveness and program ranking, size, affiliation, and location. Reference Daniel, Vila, Leon Guerrero and Karroum4 In contrast, our study examines program websites in the setting of COVID-19 when websites became increasingly significant due to restrictions to in-person experiences. While data were collected from February-April 2020 in the previous study, Reference Daniel, Vila, Leon Guerrero and Karroum4 one-third of residency online resources were created after March 2020. Reference Singh, DeAtkine and Hattaway12 Consequently, our evaluation in 2022 captured almost 20 additional websites. Moreover, our study compares Canadian and US program websites and incorporates more extensive criteria, such as the community and environment, which may be crucial to applicants who have not visited the program location.
Our study adds to the literature by examining neurology program websites following the shift to a virtual platform due to COVID-19. Additionally, it is novel in comparing American and Canadian program websites. While we used validated scoring criteria from the literature, Reference Gaeta, Birkhahn, Lamont, Banga and Bove8 our assessment may have been subjective and excluded attributes crucial to applicants. As websites are constantly updated, we were unable to reflect changes over time with cross-sectional data. To alleviate this, we collected data near the start of residency applications to capture websites at their most comprehensive state. Within Canadian regional analysis, statistical accuracy and result interpretation were limited due to modest sample sizes. Moreover, it is important to note that US program rankings are influenced by factors beyond website comprehensiveness, such as hospital resources and staff training opportunities.
Our study highlights components emphasized by neurology residency training programs on their websites within the context of COVID-19 and underlines key differences between American and Canadian residency program websites. Based on our experience, we recommend the following as a minimum standard for a comprehensive program website: information regarding recruitment (e.g., program description, contact information), faculty and residents (e.g., name list, photos), education (e.g., curriculum, rotations), teaching (e.g., grand rounds, research requirement), benefits (e.g., salary, vacation/leave policies), and community/environment (e.g., hospital sites, local attractions). Future studies in website design and interface could determine other aspects important to applicants. As interviews remain virtual, comprehensive program websites are vital in helping applicants make informed decisions and select the most suitable program.
Table 2: Evaluation scores of adult neurology residency program websites in the United States (US) (N = 171 a ) and Canada (N = 16)
a Three of 174 US neurology residency programs had inaccessible websites.
b Includes criteria from all domains. SD, standard deviation.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/cjn.2023.266.
Author contributions
WW and BT contributed equally to this work. CT and FK designed the work. CT and WW wrote the main manuscript text. CT and WW conducted all data collection. BT conducted the statistical data analysis. TC, SM, MTK, and FK substantively revised the work. All authors reviewed and approved the final manuscript prior to submission. All authors agree to be accountable for their contributions and ensure all inquiries related to the work are resolved with documentation.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. The authors did not have any relationship with organizations or individuals that may have influenced this study. Dr Faisal Khosa is the recipient of the Michael Smith Health Research BC Health Professional-Investigator award (2023-2028); Don Rix Physician Leadership Lifetime Achievement Award (2022); BC Achievement Foundation – Mitchell Award of Distinction (2022); University of British Columbia – Distinguished Achievement Award for Equity, Diversity & Inclusion (2022) and Vancouver Medical Dental & Allied Staff Association – Equity, Diversity & Inclusion Award (2022).
Competing interests
None.
IRB approval status
This study was exempt from institutional review board approval as the data was extracted in its entirety from publicly available resources. The dissemination of results will not identify any individual or generate new forms of identifiable information.
Open access
