Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-586b7cd67f-l7hp2 Total loading time: 0 Render date: 2024-11-28T16:40:40.343Z Has data issue: false hasContentIssue false

Chapter 11 - Using Online Survey Tools to Improve Access to International Experts

The ‘E-Delphi’

Published online by Cambridge University Press:  14 November 2024

By
Georgina Clutterbuck
Edited by
Paul M.W. Hackett ,
Christopher M. Hayre ,
Dave Muller ,
Marcia Scherer  and
Ava Gordley-Smith
Paul M.W. Hackett
Affiliation:
Emerson College, Boston
Christopher M. Hayre
Affiliation:
The University of Canberra
Dave Muller
Affiliation:
Suffolk University, Massachusetts
Marcia Scherer
Affiliation:
University of Rochester Medical Center, New York
Ava Gordley-Smith
Affiliation:
University of Suffolk
Get access

Summary

The Delphi technique has been used since the 1950s to reach consensus between experts through a series of iterative questionnaires.  Delphi studies were traditionally conducted in-person, or via ‘snail-mail’. Early iterations of the ‘e-Delphi’ mimicked these traditional methods by employing technological communication processes such as e-mail. The increasing availability and useability of online survey tools has modernised the Delphi process. This chapter explores the benefits of conducting a Delphi study in an online environment using modern survey tools. Experiences with the e-Delphi during research in areas such as rehabilitation sciences have indicated benefits including the ability to recruit experts regardless of their location, improved participant anonymity, streamlined data analysis, and substantial time and cost savings. However, while online methods can diminish geographical inequity, restricted digital infrastructure access and gaps in recruiter knowledge of potential participants can simultaneously increase discrimination and bias.

Keywords

e-Delphionline surveyelectronic Delphiexpert consensusonline Delphi
Type
Chapter
Information
The Future of Qualitative Research in Healthcare
The Role and Management of Digital Methods
 , pp. 161 - 184
Publisher: Cambridge University Press
Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adler, M., & Ziglio, E. (1996). Gazing Into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health. Jessica Kingsley Publishers.Google Scholar
Bäck-Pettersson, S., Hermansson, E., Sernert, N., & Björkelund, C. (2008). Research Priorities in Nursing: A Delphi Study among Swedish Nurses. Journal of Clinical Nursing, 17(16), 2221–31. https://doi.org/10.1111/j.1365-2702.2007.02083.x.CrossRefGoogle Scholar
Barrett, D., & Heale, R. (2020). What are Delphi studies? Evidence Based Nursing, 23(3), 68–9. https://doi.org/10.1136/ebnurs-2020-103303.CrossRefGoogle ScholarPubMed
Best, R. J. (1974). An Experiment in Delphi Estimation in Marketing Decision Making. Journal of Marketing Research, 11(4), 448–52. https://doi.org/10.2307/3151295.CrossRefGoogle Scholar
Birko, S., Dove, E. S., Özdemir, V., & Dalal, K. (2015). Evaluation of Nine Consensus Indices in Delphi Foresight Research and their Dependency on Delphi Survey Characteristics: A Simulation Study and Debate on Delphi Design and Interpretation. PLoS One, 10(8), e0135162–e0135162. https://doi.org/10.1371/journal.pone.0135162.CrossRefGoogle ScholarPubMed
Buck, A. J., Gross, M., Hakim, S., & Weinblatt, J. (1993). Using the Delphi Process to Analyze Social Policy Implementation: A Post Hoc Case from Vocational Rehabilitation. Policy Sciences, 26(4), 271–88. https://doi.org/10.1007/BF00999473.CrossRefGoogle Scholar
Culley, J. M. (2011). Use of a Computer-Mediated Delphi Process to Validate a Mass Casualty Conceptual Model. Computers, Informatics, Nursing, 29(5), 272–9. https://doi.org/10.1097/NCN.0b013e3181fc3e59.CrossRefGoogle Scholar
Dalkey, N., & Helmer, O. (1963). An Experimental Application of the DELPHI Method to the Use of Experts. Management Science, 9(3), 458–67. https://doi.org/10.1287/mnsc.9.3.458.CrossRefGoogle Scholar
de Villiers, M. R., de Villiers, P. J. T., & Kent, A. P. (2005). The Delphi Technique in Health Sciences Education Research. Medical Teacher, 27(7), 639–43. https://doi.org/10.1080/01421590500069947.CrossRefGoogle ScholarPubMed
Duffield, C. (1993). The Delphi Technique: A Comparison of Results Obtained using Two Expert Panels. International Journal of Nursing Studies, 30(3), 227–37. https://doi.org/10.1016/0020-7489(93)90033-Q.CrossRefGoogle ScholarPubMed
Education and Outreach Working Group & Accessibility Guidelines Working Group (2005). WCAG 2 Overview. www.w3.org/TR/WCAG20/.Google Scholar
Forbes, R., Mandrusiak, A., Smith, M., & Russell, T. (2017). Identification of Competencies for Patient Education in Physiotherapy using a Delphi Approach. Physiotherapy, 104(2), 232–8. https://doi.org/10.1016/j.physio.2017.06.002.Google ScholarPubMed
Giltenane, M., Sheridan, A., Kroll, T., & Frazer, K. (2022). Identification of Quality Indicators of Public Health Nursing Practice: ‘Modified Delphi’ Approach. Public Health Nursing, 39(1), 214–28. https://doi.org/10.1111/phn.13000.CrossRefGoogle ScholarPubMed
Goodman, C. M. (1987). The Delphi Technique: A Critique. Journal of Advanced Nursing, 12(6), 729–34. https://doi.org/10.1111/j.1365-2648.1987.tb01376.x.CrossRefGoogle Scholar
Higgins, J. M., Arnold, S. R. C., Weise, J., Pellicano, E., & Trollor, J. N. (2021). Defining Autistic Burnout through Experts by Lived Experience: Grounded Delphi Method Investigating #AutisticBurnout. Autism, 25(8), 2356–69. https://doi.org/10.1177/13623613211019858.CrossRefGoogle ScholarPubMed
Hult Khazaie, D., & Khan, S. S. (2020). Social Psychology and Pandemics: Exploring Consensus about Research Priorities and Strategies using the Delphi Method. Asian Journal of Social Psychology, 23(4), 363–71. https://doi.org/10.1111/ajsp.12442.CrossRefGoogle Scholar
Hung, H.-L., Altschuld, J. W., & Lee, Y.-F. (2008). Methodological and Conceptual Issues Confronting a Cross-Country Delphi Study of Educational Program Evaluation. Evaluation and Program Planning, 31(2), 191–8. https://doi.org/10.1016/j.evalprogplan.2008.02.005.CrossRefGoogle ScholarPubMed
Jenkins, D. A., & Smith, T. E. (1994). Applying Delphi Methodology in Family Therapy Research. Contemporary Family Therapy, 16(5), 411–30. https://doi.org/10.1007/BF02197902.CrossRefGoogle Scholar
Keeney, S., Hasson, F., & McKenna, H. P. (2011). The Delphi Technique in Nursing and Health Research. Chichester: Wiley-Blackwell.CrossRefGoogle Scholar
Lewis-Beck, M. S., Bryman, A., & Liao, T. F. (2003). The SAGE Encyclopedia of Social Science Research Methods (Vol. 3). Thousand Oaks: SAGE Publications.Google Scholar
Linstone, H. A., & Turoff, M. (1975). The Delphi Method: Techniques and Applications. Reading, MA: Addison-Wesley Pub. Co., Advanced Book Program.Google Scholar
Linstone, H. A., & Turoff, M. (2011). Delphi: A Brief Look Backward and Forward. Technological Forecasting & Social Change, 78(9), 1712–19. https://doi.org/10.1016/j.techfore.2010.09.011.CrossRefGoogle Scholar
Mackenzie, L., Coppola, S., Alvarez, L., et al. (2017). International Occupational Therapy Research Priorities: A Delphi Study. OTJR (Thorofare, N.J.), 37(2), 72–81. https://doi.org/10.1177/1539449216687528.Google Scholar
Markovic, D. S., Zivkovic, D., Branovic, I., Popovic, R., & Cvetkovic, D. (2013). Smart Power Grid and Cloud Computing. Renewable & Sustainable Energy Reviews, 24, 566–77. https://doi.org/10.1016/j.rser.2013.03.068.CrossRefGoogle Scholar
McIntyre, S., Novak, I., & Cusick, A. (2010). Consensus Research Priorities for Cerebral Palsy: A Delphi Survey of Consumers, Researchers, and Clinicians. Developmental Medicine & Child Neurology, 52(3), 270–5. https://doi.org/10.1111/j.1469-8749.2009.03358.x.CrossRefGoogle Scholar
McKenna, H. P. (1994). The Delphi Technique: A Worthwhile Research Approach for Nursing? Journal of Advanced Nursing, 19(6), 1221–5. https://doi.org/10.1111/j.1365-2648.1994.tb01207.x.CrossRefGoogle ScholarPubMed
Okoli, C., & Pawlowski, S. D. (2004). The Delphi Method as a Research Tool: An Example, Design Considerations and Applications. Information & Management, 42(1), 15–29. https://doi.org/10.1016/j.im.2003.11.002.CrossRefGoogle Scholar
Palisano, R. J., Rosenbaum, P., Bartlett, D., & Livingston, M. H. (2008). Content Validity of the Expanded and Revised Gross Motor Function Classification System. Developmental Medicine & Child Neurology, 50(10), 744–50. https://doi.org/10.1111/j.1469-8749.2008.03089.x.CrossRefGoogle ScholarPubMed
Rahimzadeh, V., Bartlett, G., & Knoppers, B. M. (2021). A Policy Delphi Study to Validate the Key Implications of Data Sharing (KIDS) Framework for Pediatric Genomics in Canada. BMC Medical Ethics, 22(1), 1–71. https://doi.org/10.1186/s12910-021-00635-1.CrossRefGoogle ScholarPubMed
Rankin, G., Rushton, A., Olver, P., & Moore, A. (2012). Chartered Society of Physiotherapy’s Identification of National Research Priorities for Physiotherapy using a Modified Delphi Technique. Physiotherapy, 98(3), 260–72. https://doi.org/10.1016/j.physio.2012.03.002.CrossRefGoogle Scholar
Richards, K. L., Woolrych, I., Allen, K. L., & Schmidt, U. (2022). A Delphi Study to Explore Clinician and Lived Experience Perspectives on Setting Priorities in Eating Disorder Services. BMC Health Services Research, 22(1), 1–788. https://doi.org/10.1186/s12913-022-08170-4.CrossRefGoogle ScholarPubMed
Rowe, G., & Wright, G. (1999). The Delphi Technique as a Forecasting Tool: Issues and Analysis. International Journal of Forecasting, 15(4), 353–75. https://doi.org/10.1016/S0169-2070(99)00018-7.CrossRefGoogle Scholar
Rowe, G., & Wright, G. Expert Opinions in Forecasting: The Role of the Delphi Technique. In Principles of Forecasting (pp. 125–44). Boston, MA: Springer US. https://doi.org/10.1007/978-0-306-47630-3_7.CrossRefGoogle Scholar
Rowe, G., Wright, G., & McColl, A. (2005). Judgment Change during Delphi-like Procedures: The Role of Majority Influence, Expertise, and Confidence. Technological Forecasting & Social Change, 72(4), 377–99. https://doi.org/10.1016/j.techfore.2004.03.004.CrossRefGoogle Scholar
Sackman, H. (1974). Delphi Critique: Expert Opinion, Forecasting, and Group Process. Lexington: Lexington Books.Google Scholar
Sole, G., Skinner, M., Hale, L., & Golding, C. (2019). Developing a Framework for Teaching Clinical Reasoning Skills to Undergraduate Physiotherapy Students: A Delphi Study. New Zealand Journal of Physiotherapy, 47(1), 49–58. https://doi.org/10.15619/NZJP/47.1.06.CrossRefGoogle Scholar
Sun, C., Dohrn, J., Klopper, H., et al. (2015). Clinical Nursing and Midwifery Research Priorities in Eastern and Southern African Countries: Results From a Delphi Survey. Nursing Research, 64(6), 466–75. https://doi.org/10.1097/NNR.0000000000000126.CrossRefGoogle ScholarPubMed
Thorpe, R., & Holt, R. (2007). The SAGE Dictionary of Qualitative Management Research. London: SAGE Publications, Limited.Google Scholar
Verschuren, O., Ketelaar, M., Keefer, D., et al. (2011). Identification of a Core Set of Exercise Tests for Children and Adolescents with Cerebral Palsy: A Delphi Survey of Researchers and Clinicians. Developmental Medicine & Child Neurology, 53(5), 449–56. https://doi.org/10.1111/j.1469-8749.2010.03899.x.CrossRefGoogle ScholarPubMed
Winkler, J., & Moser, R. (2016). Biases in Future-Oriented Delphi Studies: A Cognitive Perspective. Technological Forecasting & Social Change, 105, 63–76. https://doi.org/10.1016/j.techfore.2016.01.021.CrossRefGoogle Scholar
World Bank. (2022). Individuals using the Internet (% of population). https://data.worldbank.org/indicator/IT.NET.USER.ZS.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×