Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-q99xh Total loading time: 0 Render date: 2024-12-18T22:07:56.216Z Has data issue: false hasContentIssue false

Chapter 14 - Algorithms as Co-Researchers

Exploring Meaning and Bias in Qualitative Research

from Part III - Illustrative Examples and Emergent Issues

Published online by Cambridge University Press:  08 June 2023

By
Wendy Arianne Günther ,
Mark Thompson ,
Mayur P. Joshi  and
Stavros Polykarpou
Edited by
Boyka Simeonova  and
Robert D. Galliers
Boyka Simeonova
Affiliation:
University of Leicester
Robert D. Galliers
Affiliation:
Bentley University, Massachusetts and Warwick Business School
Get access

Summary

This chapter discusses the reflexive relationship between qualitative researchers and the process of selecting, forming, processing and interpreting data in algorithmic qualitative research. Drawing on Heidegger’s ideas, it argues that such research is necessarily synthetic – even creative – in that these activities inflect, and are in turn inflected by, the data itself. Thus, methodological transparency is key to understanding how different types of meanings become infused in the process of algorithmic qualitative research. While algorithmic research practices provide multiple opportunities for creating transparent meaning, researchers are urged to consider how such practices can also introduce and reinforce human and algorithmic bias in the form of unacknowledged introduction of perspectives into the data. The chapter demonstrates this reflexive dance of meaning and bias using an illustrative case of topic modelling. It closes by offering some recommendations for engaging actively with the domain, considering a multi-disciplinary approach, and adopting complementary methods that could potentially help researchers in fostering transparency and meaning.

Keywords

algorithmic qualitative researchalgorithmstransparencymeaningopacitybiastopic modellingmethodologydata
Type
Chapter
Information
Cambridge Handbook of Qualitative Digital Research , pp. 211 - 228
Publisher: Cambridge University Press
Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abbasi, A., Sarker, S. and Chiang, R. (2016). Big data research in information systems: Toward an inclusive research agenda. Journal of the Association for Information Systems, 17(2), i–xxxii.Google Scholar
Agrawal, A., Fu, W. and Menzies, T. (2018). What is wrong with topic modeling? And how to fix it using search-based software engineering. Information and Software Technology, 98, 74–88.Google Scholar
Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired, www.wired.com/2008/06/pb-theory/.Google Scholar
Avgerou, C. (2019). Contextual explanation: Alternative approaches and persistent challenges. MIS Quarterly, 43(3), 977–1006.Google Scholar
Bao, Y. and Datta, A. (2014). Simultaneously discovering and quantifying risk types from textual risk disclosures. Management Science, 60(6), 1371–1391.CrossRefGoogle Scholar
Bapna, S., Benner, M. J. and Qiu, L. (2019). Nurturing online communities: An empirical investigation. MIS Quarterly, 43(2), 425–452.CrossRefGoogle Scholar
Barthes, R. (1972). The Structuralist Activity: Critical Essays. Evanston, IL: Northwestern University Press.Google Scholar
Barrett, M. and Oborn, E. (2018). Bridging the research-practice divide: Harnessing expertise collaboration in making a wider set of contributions. Information and Organization, 28(1), 44–51.Google Scholar
Belford, M., Mac Namee, B. and Greene, D. (2018). Stability of topic modeling via matrix factorization. Expert Systems with Applications, 91, 159–169.Google Scholar
Berry, M. W., Browne, M., Langville, A. N., Pauca, B. P. and Plemmons, R. J. (2007). Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analytics, 52(1), 155–173.Google Scholar
Binkley, D., Heinz, D., Lawrie, D. and Overfelt, J. (2014). Understanding LDA in source code analysis. Proceedings of the International Conference on Program Comprehension, Hyderabad, India.Google Scholar
Bird, S., Klein, E. and Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. Sebastopol, CA: O’Reilly Media, Inc.Google Scholar
Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77–84.CrossRefGoogle Scholar
Blei, D. M. and Lafferty, J. D. (2009). Topic models. In Srivastava, A. N. and Sahami, M. (eds), Text Mining: Classification, Clustering, and Applications. London: Chapman & Hall, pp. 71–94.Google Scholar
Brynjolfsson, E. and McAfee, A. (2017). The business of artificial intelligence. Harvard Business Review, 7, 1–20.Google Scholar
Busa, R. (1990). Informatics and the new philology. Computers and the Humanities, 24(5–6), 339–343.Google Scholar
Chang, J., Gerrish, S., Wang, C., Boyd-Graber, J. L. and Blei, D. M. (2009). Reading tea leaves: How humans interpret topic models. Advances in Neural Information Processing Systems, 22, 288–296.Google Scholar
Chen, L., Baird, A. and Straub, D. W. (2019). An analysis of the evolving intellectual structure of health information systems research in the information systems discipline. Journal of the Association for Information Systems, 20(8), 1023–1074.Google Scholar
Christin, A. (2020). Algorithmic ethnography, during and after COVID-19. Communication and the Public, 5(3–4), 108–111.Google Scholar
Creswell, J. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2nd edition. Los Angeles, CA: Sage.Google Scholar
Croidieu, G. and Kim, P. H. (2018). Labor of love: Amateurs and lay-expertise legitimation in the early U.S. radio field. Administrative Science Quarterly, 63(1), 1–42.Google Scholar
Dong, W., Liao, S. and Zhang, Z. (2018). Leveraging financial social media data for corporate fraud detection. Journal of Management Information Systems, 35(2), 461–487.Google Scholar
Dreyfus, H. (1993). What Computers Still Can’t Do: A Critique of Artificial Reason, 2nd edition. Cambridge, MA: MIT Press.Google Scholar
Dubé, L. and Paré, G. (2003). Rigor in information systems positivist case research: Current practices, trends, and recommendations. MIS Quarterly, 27(4), 597–636.CrossRefGoogle Scholar
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T. et al. (2021). Artificial intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management, 57, 1–47.Google Scholar
Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550.Google Scholar
Evangelopoulos, N. (2016). Thematic orientation of the ISJ within a semantic space of IS research. Information Systems Journal, 26(1), 39–46.Google Scholar
Evangelopoulos, N., Zhang, X. and Prybutok, V. R. (2012). Latent semantic analysis: five methodological recommendations. European Journal of Information Systems, 21(1), 70–86.CrossRefGoogle Scholar
Galdas, P. (2017). Revisiting bias in qualitative research: Reflections on its relationship with funding and impact. International Journal of Qualitative Methods, 16(1), 1–2.Google Scholar
George, G., Haas, M. R. and Pentland, A. (2014). Big data and management. Academy of Management Journal, 57(2), 321–326.Google Scholar
Geva, H., Oestreicher-Singer, G. and Saar-Tsechansky, M. (2019). Using retweets when shaping our online persona: Topic modeling approach. MIS Quarterly, 43(2), 501–524.Google Scholar
Giorgi, S. and Weber, K. (2015). Marks of distinction: Framing and audience appreciation in the context of investment advice. Administrative Science Quarterly, 60(2), 333–367.Google Scholar
Giorgi, S., Maoret, M. and Zajac, E. J. (2019). On the relationship between firms and their legal environment: The role of cultural consonance. Organization Science, 30(4), 803–830.Google Scholar
Glaser, V. L., Pollock, N. and D’Adderio, L. (2021). The biography of an algorithm: Performing algorithmic technologies in organizations. Organization Theory, 2(2), 1–27.Google Scholar
Goes, P. B. (2014). Editor’s comments: Big data and IS research. MIS Quarterly, 38(3), iii–viii.Google Scholar
Gong, J., Abhishek, V. and Li, B. (2018). Examining the impact of keyword ambiguity on search advertising performance: A topic model approach. MIS Quarterly, 42(3), 805–829.CrossRefGoogle Scholar
Günther, W. and Joshi, M. (2020). Algorithmic intelligence in research: Prevalent topic modeling practices and implications for rigor in IS and management research. International Conference on Information Systems, virtual conference.Google Scholar
Haans, R. F. (2019). What’s the value of being different when everyone is? The effects of distinctiveness on performance in homogeneous versus heterogeneous categories. Strategic Management Journal, 40(1), 3–27.Google Scholar
Hannigan, T., Haans, R. F., Vakili, K., Tchalian, H., Glaser, V., Wang, M. et al. (2019). Topic modeling in management research: Rendering new theory from textual data. Academy of Management Annals, 13(2), 586–632.Google Scholar
Heidegger, M. (1962). Being and Time. Oxford, UK and Cambridge, MA: Blackwell.Google Scholar
Heidegger, M. (1977). The question concerning technology. In Krell, D. F. (ed.), Basic Writings. New York, NY: HarperCollins, pp. 213–238.Google Scholar
Hickman, L., Thapa, S., Tay, L., Cao, M. and Srinivasan, P. (2022). Text preprocessing for text mining in organizational research: Review and recommendations. Organizational Research Methods, 25(1), 114–146.Google Scholar
Huang, A. H., Lehavy, R., Zang, A. Y. and Zheng, R. (2018). Analyst information discovery and interpretation roles: A topic modeling approach. Management Science, 64(6), 2833–2855.Google Scholar
Introna, L. and Ilharco, F. (2011). Phenomenology, screens, and screenness: Returning to the world itself. In Galliers, R. D. and Currie, W. (eds), The Oxford Handbook of Management Information Systems: Critical Perspectives and New Directions. Oxford: Oxford University Press.Google Scholar
Jarzabkowski, P., Bednarek, R. and Lê, J. K. (2014). Producing persuasive findings: Demystifying ethnographic textwork in strategy and organization research, Strategic Organization, 12(4), 274–287.Google Scholar
Jones, M. (2019). What we talk about when we talk about (big) data. Journal of Strategic Information Systems, 28(1), 3–16.Google Scholar
Jones, S. (2016). Roberto Busa, S. J., and the Emergence of Humanities Computing: The Priest and the Punched Cards. London: Routledge.Google Scholar
Kaplan, S. and Vakili, K. (2015). The double‐edged sword of recombination in breakthrough innovation. Strategic Management Journal, 36(10), 1435–1457.Google Scholar
Karanović, J., Berends, H. and Engel, Y. (2020). Regulated dependence: Platform workers’ responses to new forms of organizing. Journal of Management Studies, 58(4), 1070–1106.Google Scholar
Kitchin, R. (2014). Big data, new epistemologies and paradigm shifts. Big Data Society, 1(1), 1–12.Google Scholar
Klein, H. K. and Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23(1), 67–93.Google Scholar
Larsen, K. R. and Bong, C. H. (2016). A tool for addressing construct identity in literature reviews and meta-analyses. MIS Quarterly, 40(3), 529–551.Google Scholar
Lash, M. T. and Zhao, K. (2016). Early predictions of movie success: The who, what, and when of profitability. Journal of Management Information Systems, 33(3), 874–903.Google Scholar
Latour, B. (1990). Technology is society made durable. Sociological Review, 38(Supp. 1), 103–131.Google Scholar
Lee, G. M., Qiu, L. and Whinston, A. B. (2016). A friend like me: Modeling network formation in a location-based social network. Journal of Management Information Systems, 33(4), 1008–1033.Google Scholar
Mackenzie, A. (2017). Machine Learners: Archaeology of a Data Practice. Cambridge, MA: MIT Press.Google Scholar
Madsen, A. K. (2015). Between technical features and analytic capabilities: Charting a relational affordance space for digital social analytics. Big Data & Society, 2(1), 1–15.Google Scholar
Manovich, L (2011). Trending: The promises and the challenges of big social data. Manovich, http://manovich.net/index.php/projects/trending-the-promises-and-the-challenges-of-big-social-data.Google Scholar
McCarty, W. (2003). Humanities computing, www.mccarty.org.uk/essays/McCarty,%20Humanities%20computing.pdf.Google Scholar
Newell, S. and Marabelli, M. (2015). Strategic opportunities (and challenges) of algorithmic decision-making: A call for action on the long-term societal effects of ‘datification’. Journal of Strategic Information Systems, 24(1), 3–14.Google Scholar
Nielsen, M. W. and Börjeson, L. (2019). Gender diversity in the management field: Does it matter for research outcomes? Research Policy, 48(7), 1617–1632.Google Scholar
Pääkkönen, J. and Ylikoski, P. (2021). Humanistic interpretation and machine learning. Synthese, 199(1), 1461–1497.Google Scholar
Patton, M. Q. (2002). Qualitative Research & Evaluation Methods, 3rd edition. Thousand Oaks, CA: Sage.Google Scholar
Ramsay, S. (2011). Reading Machines: Toward an Algorithmic criticism. Champaign, IL: University of Illinois Press.Google Scholar
Ribes, D. and Jackson, S. J. (2013). Data bite man: The work of sustaining long-term study. In Gitelman, L. (ed.), ‘Raw Data’ Is an Oxymoron. Cambridge, MA: MIT Press, pp. 147–166.Google Scholar
Roberts, M. E., Stewart, B. M., Tingley, D. and Airoldi, E. M. (2013). The structural topic model and applied social science. Advances in Neural Information Processing Systems Workshop on Topic Models: Computation, Application, and Evaluation, Lake Tahoe, NV.Google Scholar
Rockwell, G. and Sinclair, S. (2016). Computer-Assisted Interpretation in the Humanities. Cambridge, MA: MIT Press.Google Scholar
Salge, T. O., Antons, D., Barrett, M., Kohli, R., Oborn, E. and Polykarpou, S. (2022). How IT investments help hospitals gain and sustain reputation in the media: The role of signaling and framing. Information Systems Research, 33(1), 110–130.Google Scholar
Samtani, S., Chinn, R., Chen, H. and NunamakerJr, J. F. (2017). Exploring emerging hacker assets and key hackers for proactive cyber threat intelligence. Journal of Management Information Systems, 34(4), 1023–1053.Google Scholar
Schmiedel, T., Müller, O. and vom Brocke, J. (2019). Topic modeling as a strategy of inquiry in organizational research: A tutorial with an application example on organizational culture. Organizational Research Methods, 22(4), 941–968.Google Scholar
Schultze, U. and Avital, M. (2011). Designing interviews to generate rich data for information systems research. Information and Organization, 21(1), 1–16.CrossRefGoogle Scholar
Shadish, W. R., Cook, T. D. and Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton, Mifflin & Co., Vol. XXI.Google Scholar
Shi, Z., Lee, G. M. and Whinston, A. B. (2016). Toward a better measure of business proximity: Topic modeling for industry intelligence. MIS Quarterly, 40(4), 1035–1056.Google Scholar
Shi, D., Guan, J., Zurada, J. and Manikas, A. (2017). A data-mining approach to identification of risk factors in safety management systems. Journal of Management Information Systems, 34(4), 1054–1081.Google Scholar
Sidorova, A., Evangelopoulos, N., Valacich, J. S. and Ramakrishnan, T. (2008). Uncovering the intellectual core of the information systems discipline. MIS Quarterly, 32(3), 467–482.CrossRefGoogle Scholar
Suchman, L. (2006). Human-Machine Reconfigurations. Cambridge: Cambridge University Press.Google Scholar
Suchman, L. (1987). Plans and Situated Actions: The Problem of Human-Machine Communication (Learning in Doing: Social, Cognitive and Computational Perspectives). Cambridge: Cambridge University Press.Google Scholar
Tashakkori, A. and Creswell, J. W. (2007). Exploring the nature of research questions in mixed methods research. Journal of Mixed Methods Research, 1(3), 207–211.Google Scholar
Tracy, S. J. (2013). Qualitative Research Methods: Collecting Evidence, Crafting Analysis, Communicating Impact. Chichester, UK: Wiley-Blackwell.Google Scholar
Tully, J. (1994). Philosophy in an Age of Pluralism. Cambridge: Cambridge University Press.Google Scholar
Van Maanen, J. (2011). Ethnography as work: Some rules of engagement. Journal of Management Studies, 48(1), 218–234.Google Scholar
Van Maanen, J. and de Rond, M. (2017). The making of a classic ethnography: Notes on Alice Goffman’s On the Run. Academy of Management Review, 42(2), 396–406.Google Scholar
Weber, M. (2003). The Protestant Ethic and the Spirit of Capitalism. New York, NY: Courier Dover.Google Scholar
Wendt, T. (2013). Designing for transparency and the myth of the modern interface. Ux Magazine, https://uxmag.com/articles/designing-for-transparency-and-the-myth-of-the-modern-interface.Google Scholar
Winner, M. (1999). Do artifacts have politics? In MacKenzie, D. A. and Wajcman, J. (eds), The Social Shaping of Technology, 2nd edition. London: Open University Press, pp. 28–40. (Originally printed in Daedalus, 109(1), winter 1980.)Google Scholar
Winograd, T. and Flores, F. (1986). Understanding Computers and Cognition: A New Foundation for Design. Norwood, NJ: Ablex.Google Scholar
Yin, R. K. (1981). The case study crisis: Some answers. Administrative Science Quarterly, 26(1), 58–65.Google Scholar
Yue, W. T., Wang, Q. H. and Hui, K. L. (2019). See no evil, hear no evil? Dissecting the impact of online hacker forums. MIS Quarterly, 43(1), 73–95.Google Scholar
Zammuto, R. F., Griffith, T. L., Majchrzak, A., Dougherty, D. J. and Faraj, S. (2007). Information technology and the changing fabric of organization. Organization Science, 18(5), 749–762.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×