Hostname: page-component-cd9895bd7-fscjk Total loading time: 0 Render date: 2024-12-19T04:04:52.623Z Has data issue: false hasContentIssue false
  • English
  • Français

Curtailing data biases in business research: Introducing a hybrid approach

Published online by Cambridge University Press:  26 February 2018

Muhammad Aftab Alam  and
Omar Khalid Bhatti
Muhammad Aftab Alam*
Affiliation:
Department of Marketing and Management, Macquarie University, Sydney, NSW, Australia
Omar Khalid Bhatti
Affiliation:
College of Business, Antaliya Bilm University, Antalya, Turkey
*
Corresponding author: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

This study elucidates the falsity of business research in relying on either respondents or informants alone for data collection, and argues that with the biased data, business research cannot provide unbiased solutions. We compare 400 reports (200 respondents and 200 informants) on the workplace deviance and assess the goodness of both the techniques. Analysis of variance and posthoc (descriptive discriminant analysis) indicate significant disparities between the two approaches across all items. In the informant’s role, people tend to overreport, whereas in the respondent’s part they underreport an undesirable behavior. Further, we find that conventional techniques for assessing the construct’s validity and common-method bias neither assures realistic measurement nor eliminate the response bias. Drawing on the theory of psychological projection, we propose a hybrid approach that curtails some of the main biases in data and measurement. Qualitative confirmation through informal interviews with managers in the investigated firms validates the proposed method.

Keywords

databiasinformantrespondentmethodresearch
Type
Research Article
Information
Journal of Management & Organization , Volume 28 , Issue 4 , July 2022 , pp. 909 - 923
Copyright
Copyright © Cambridge University Press and Australian and New Zealand Academy of Management 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allport, F. H. (1924). The group fallacy in relation to social science. American Journal of Sociology, 29(6), 688–706.CrossRefGoogle Scholar
Allport, D. A. (1968). Phenomenal simultaneity and the perceptual moment hypothesis. British Journal of Psychology, 59(4), 395–406.CrossRefGoogle Scholar
Ariely, D., Gneezy, U., Loewenstein, G., & Mazar, N. (2009). Large stakes and big mistakes. The Review of Economic Studies, 76(2), 451–469.CrossRefGoogle Scholar
Becker, T. E., & Vance, R. J. (1992). Construct validity of three types of organisational citizenship behaviour using a direct product model. Paper presented at the 1992 Academy of Management Meeting, Las Vegas, NV, August 11.Google Scholar
Bell, C. M., & Hughes-Jones, J. (2007). Power, self-regulation and the moralization of behavior. Journal of Business Ethics, 83(3), 503–514.CrossRefGoogle Scholar
Bendle, N. T., & Wang, X. S. (2016). Uncovering the message from the mess of big data. Business Horizons, 59(1), 115–124.CrossRefGoogle Scholar
Bennett, R. J., & Robinson, S. L. (2000). Development of a measure of workplace deviance. Journal of Applied Psychology, 85(3), 349–360.CrossRefGoogle ScholarPubMed
Bentley, T. A., Catley, B. E., Forsyth, D., & Tappin, D. C. (2013). Understanding workplace violence and its prevention in New Zealand: The 2011 New Zealand workplace violence survey. Journal of Management & Organization, 19(3), 352–364.CrossRefGoogle Scholar
Bernard, H. R., Killworth, P., Kronenfeld, D., & Sailer, L. (1984). The problem of informant accuracy: The validity of retrospective data. Annual Review of Anthropology, 13(1), 495–517.CrossRefGoogle Scholar
Borman, W. C. (1991). Job behavior, performance, and effectiveness. In M. D. Dunnette, & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 271–326). Newbury Park, CA: Sage.Google Scholar
Brewer, N., Mitchell, P., & Weber, N. (2002). Gender role, organizational status and conflict management styles. The International Journal of Conflict Management, 13(1), 78–94.Google Scholar
Brutus, S., Gill, H., & Duniewicz, K. (2013). State-of-science in industrial and organizational psychology: A review of self-reported limitations. Journal of Management, 39(1), 48e75.Google Scholar
Campbell, D. T. (1955). The informant in quantitative research. American Journal of Sociology, 60(4), 339–342.CrossRefGoogle Scholar
Campbell, D. T. & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81.CrossRefGoogle Scholar
Chen, M. J., Farh, J. L., & MacMillan, I. C. (1993). An exploration of the expertness of outside informants. Academy of Management Journal, 36(6), 1614–1632.CrossRefGoogle Scholar
Cronbach, L. J. & Meehl, P. E. (1955). Construct validity in psychological tests Psychological Bulletin, 52(4), 281.CrossRefGoogle Scholar
Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966.CrossRefGoogle ScholarPubMed
Cusumano, M. A., & Takeishi, A. (1991). Supplier relations and management: A survey of Japanese, Japanese-transplant, and U.S. auto plants. Strategic Management Journal, 12, 563–588.CrossRefGoogle Scholar
Dolnicar, S., & Grun, B. (2009). Response style contamination of student evaluation data. Journal of Marketing Education, 31(2), 160–172.CrossRefGoogle Scholar
Donaldson, S. I., & Grant-Vallone, E. J. (2002). Understanding self-report bias in organizational behavior research. Journal of Business and Psychology, 17(2), 245–260.CrossRefGoogle Scholar
Edwards, A. L. (1957). The social desirability variable in personality assessment and research. New York: Dryden Press.Google Scholar
Fisher, R. J. (1993). Social desirability bias and the validity of indirect questioning. Journal of Consumer Research, 20(2), 303–315.CrossRefGoogle Scholar
Freud, S. (1890). Case Histories II (PFL 9), p. 132. Retrieved May 4, 2016, from http://en.wikipedia.org/wiki/Psychological_projection.Google Scholar
Gaskin, J. (2012). Common method bias using common latent factor – YouTube. Retrieved May 23, 2016, from https://www.youtube.com/watch?v=Y7Le5Vb7_jg.Google Scholar
Glick, W. H., Huber, G. P., Miller, C. C., Doty, D. H., & Sutcliffe, K. M. (1990). Studying changes in organizational design and effectiveness: Retrospective event histories and periodic assessments. Organization Science, 1(3), 293–312.CrossRefGoogle Scholar
Golden, B. R. (1992). Research notes. The past is the past—or is it? The use of retrospective accounts as indicators of past strategy. Academy of Management Journal, 35(4), 848–860.Google Scholar
Haidt, J., Koller, S. H., & Dias, M. G. (1993). Affect, culture, and morality, or is it wrong to eat your dog? Journal of Personality and Social Psychology, 65(4), 613–628.CrossRefGoogle ScholarPubMed
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (Vol. 6). Upper Saddle River, NJ: Pearson Prentice Hall.Google Scholar
Hambrick, D. C. (1981). Strategic awareness within top management teams. Strategic Management Journal, 2(3), 263–279.CrossRefGoogle Scholar
Harkness, J. A. (2010). Towards guidelines on survey adaptation. Paper for presentation at the XVII ISA World Congress of Sociology, Gothenburg, Sweden, July 11–17, pp.1–19.Google Scholar
Heide, J. B., & John, G. (1990). Alliances in industrial purchasing: The determinants of joint action in buyer-seller relationships. Journal of Marketing Research, 27, 24–36.CrossRefGoogle Scholar
Hershcovis, S. M., Reich, T. C., Parker, S. K., & Bozeman, J. (2012). The relationship between workplace aggression and target deviant behavior: The moderating roles of power and task interdependence. Work & Stress, 26(1), 1–20.CrossRefGoogle Scholar
Howard, G. S. (1994). Why do people say nasty things about self‐reports? Journal of Organizational Behavior, 15(5), 399–404.CrossRefGoogle Scholar
Hsieh, J. Y. (2013). Impact of individual and organizational factors on job satisfaction: A comparison of multilevel models and multiple regression models using different data arrangements. Journal of Management & Organization, 19(1), 44–59.CrossRefGoogle Scholar
Huber, G. P. (1985). Temporal stability and response-order biases in participant descriptions of organizational decisions. Academy of Management Journal, 28(4), 943–950.Google Scholar
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124.CrossRefGoogle ScholarPubMed
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.CrossRefGoogle ScholarPubMed
Kaplan, H. B. (1975). Self-attitudes and deviant behavior. Pacific Palisades, CA: Goodyear.Google Scholar
Kumar, N., Stern, L. W., & Anderson, J. C. (1993). Conducting interorganizational research using key informants. Academy of Management Journal, 36(6), 1633–1651.Google Scholar
Mersman, J. L., & Donaldson, S. I. (2000). Factors affecting the convergence of self-peer ratings on contextual and task performance. Human Performance, 13(3), 299–322.CrossRefGoogle Scholar
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2008). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 13–22.CrossRefGoogle Scholar
Nor Hayati, Z. (2006). Occupational stress and workplace deviant behavior. Doctoral Dissertation, Universiti Utara, Kedah, Malaysia.Google Scholar
Nutt, P. C. (1986). Tactics of implementation. Academy of Management Journal, 29(2), 230–261.Google Scholar
Peer, E., & Gamliel, E. (2011). Too reliable to be true? Response bias as a potential source of inflation in paper-and-pencil questionnaire reliability. Practical Assessment, Research & Evaluation, 16(9), 1–8.Google Scholar
Peticca-Harris, A., & Elias, S. R. (2016). A dynamic process model for finding informants and gaining access in qualitative research. Organizational Research Methods, 19(3), 376–401.CrossRefGoogle Scholar
Phillips, L. W. (1981). Assessing measurement error in key informant reports: A methodological note on organizational analysis in marketing. Journal of Marketing Research, 18, 395–415.CrossRefGoogle Scholar
Phillips, L. W. (1982). Explaining control losses in corporate marketing channels: An organizational analysis. Journal of Marketing Research, 19, 525–549.CrossRefGoogle Scholar
Pink, D. (2009, July). Dan Pink: The puzzle of motivation [Video file]. Retrieved April 17, 2016, from http://www.ted.com/talks/dan_pink_on_motivation?language=en.Google Scholar
Podsakof, P., MacKenzie, S., & Podsakoff, N. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569.CrossRefGoogle Scholar
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.CrossRefGoogle ScholarPubMed
Provan, K. G., & Skinner, S. J. (1989). Interorganizational dependence and control as predictors of opportunism in dealer-supplier relations. Academy of Management Journal, 32(1), 202–212.Google Scholar
Robinson, S. L., & Bennett, R. J. (1995). A typology of deviant workplace behaviors: A multidimensional scaling study. Academy of Management Journal, 38(2), 555–572.Google Scholar
Sackett, P. R., & Larson, J. R. (1990). Research strategies and tactics in industrial and organizational psychology. In M. D. Dunnette, & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 419–489). Palo Alto, CA: Consulting Psychologists Press.Google Scholar
Salancik, G. R., & Meindl, J. R. (1984). Corporate attributions as strategic illusions of management control. Administrative Science Quarterly, 29, 238–254.CrossRefGoogle Scholar
Scholderer, J., Grunert, K. G., & Brunsø, K. (2005). A procedure for eliminating additive bias from cross-cultural survey data. Journal of Business Research, 58(1), 72–78.CrossRefGoogle Scholar
Schooler, J. (2011). Unpublished results hide the decline effect. Nature, 470, 437.CrossRefGoogle ScholarPubMed
Schwartz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.CrossRefGoogle Scholar
Seidler, J. (1974). On using informants: A technique for collecting quantitative data and controlling measurement error in organization analysis. American Sociological Review, 39, 816–831.CrossRefGoogle Scholar
Shadish, W. R. (1993). Critical multiplism: A research strategy and its attendant tactics. New Directions for Program Evaluation, 60, 13–57.CrossRefGoogle Scholar
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.CrossRefGoogle ScholarPubMed
Sinha, J. B. P. (2010). Living and doing psychology. Psychology and Developing Societies, 22(1), 95–120.CrossRefGoogle Scholar
Sinha, R. B. N., & Hassan, A. (2014). Respondents versus informants method of data collection: Implications for business research. Multilingual Academic Journal of Education and Social Sciences, 2(1), 1–13.CrossRefGoogle Scholar
Spector, P. E. (1994). Using self-report questionnaires in OB research: A comment on the use of a controversial method. Journal of Organizational Behavior, 15, 385–392.CrossRefGoogle Scholar
Spector, P. E., & Fox, S. (2005). The Stressor-Emotion Model of Counterproductive Work Behavior. Washington, DC: American Psychological Association.CrossRefGoogle Scholar
Spradlin, D. (2012). Are you solving the right problem? Harvard Business Review, 90(9), 84–93.Google Scholar
Stevens, J. P. (2002). Applied multivariate statistics for the social sciences. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Stone, A. A., Bachrach, C. A., Jobe, J. B., Kurtzman, H. S., & Cain, V. S. (Eds.), (1999). The science of self-report: Implications for research and practice. Mahwah, NJ: Lawrence Erlbaum Associates.CrossRefGoogle Scholar
Sulaiman, M., & Bhatti, O. K. (2013). Workplace deviance and spirituality in Muslim organizations. Asian Social Science, 9(10), 237.CrossRefGoogle Scholar
Triandis, H. C. (1980). Introduction. In H. C. Triandis, & W. W. Lambert (Eds.), Handbook of cross-cultural psychology (Vol. 1, pp. 1–28). Boston, MA: Allyn & Bacon.Google Scholar
Triandis, H. C. (1994). Culture and social behavior. New York: Mc Graw Hill.Google Scholar
Triandis, H. C. (1995). Individualism and collectivism. Boulder, Co: West View Press.Google Scholar
Wade, C., & Tavris, C. (2000). Psychology (6th ed.). Upper Saddle River, NJ: Prentice Hall.Google Scholar
Wagenmakers, E. J., Wetzels, R., Borsboom, D., & van der Maas, H. (2011). Why psychologists must change the way they analyze their data: The case of psi. Journal of Personality and Social Psychology, 100, 426–432.CrossRefGoogle ScholarPubMed
Warne, R. T. (2014). A primer on multivariate analysis of variance (MANOVA) for behavioral scientists. Practical Assessment, Research & Evaluation, 19(17), 1–10.Google Scholar
Yüksel, A. (2017). A critique of ‘response bias’ in the tourism, travel and hospitality research. Tourism Management, 59, 376–384.CrossRefGoogle Scholar