Scientific knowledge is driven by the research questions we ask. As Bergman and Jean (2016) argue, if wage earners, contract workers, and other workers are underrepresented in our research samples, we're likely to fail to investigate phenomena of importance to these populations. By focusing primarily on salaried and managerial workers, we limit the research questions we ask and fail to consider important caveats to industrial-organizational theories. As Bergman and Jean note, we cannot assume that the experiences observed in any group will generalize to other groups. We elaborate on this argument to discuss how failing to take into account the diverse perspectives and needs of the full labor market leads us to fail to ask relevant research questions. Specifically, we discuss how considering unexplored labor market perspectives opens new questions in the area of ethics management.

As noted in the focal article (Chamorro-Premuzic, Winsborough, Sherman, & Hogan, 2016), recent advances in technology have provided new tools for talent identification; however, identification alone is not enough to win the war for talent (Chambers, Foulon, Handfield-Jones, Hankin, & Michael, 1998). Big data, social media, and mobile apps provide access to previously untapped information about individuals; unfortunately, that information has limited value if it cannot be used to attract and retain the top talent it identifies. Therefore, new tools for talent identification must also be vetted for how well they provide human resource (HR) practitioners a means for attracting top talent and retaining that talent. Although far from a silver bullet, gamification opens the door for collecting information on an individual's talent potential while also fostering organizational attraction and employee retention.

The growth in the popularity of social networking websites (SNSs) has resulted in many organizations using such platforms for organizational activities, including recruitment, employee selection, employee monitoring, and termination. However, the legality of organizations using social networking for these purposes is unclear. Although many researchers have touched on the potential legal implications of organizations using these tools (see e.g., Brown & Vaughn, 2011), no empirical studies have examined how the court system views their use in organizational settings. Thus, the current study examined federal court cases related to the use of SNSs in various employment practices.

Bergman and Jean (2016) include freelancers as one of the categories of workers who are understudied in the industrial and organizational (I-O) psychology literature. This neglect is particularly striking given the attention paid by the popular media and by politicians to the rise of the “gig economy,” comprising primarily short-term independent freelance workers (e.g., Cook, 2015; Kessler, 2014; Scheiber, 2014; Warner, 2015). This may be due in part to challenges involved in accessing and researching this population, as discussed by Bergman and Jean, but it may also arise from complexities in defining and conceptualizing freelance work, as well as from misunderstandings about the nature of the work now performed by many people who are considered freelancers. Major topics of interest to I-O psychologists such as organizational attraction, job satisfaction, and turnover may seem at first glance to lack relevance to the study of workers who are officially classified as self-employed. But there is substantial opportunity for I-O psychologists and other behaviorally oriented organizational researchers to contribute to our understanding of the growing number of people who earn all or some of their income by freelancing.

In the focal article “Getting Rid of Performance Ratings: Genius or Folly? A Debate,” two groups of authors argued the merits of performance ratings (Adler et al., 2016). Despite varied views, both sides noted the importance of including multiple raters to obtain more accurate performance ratings. As the pro side noted, “if ratings can be pooled across many similarly situated raters, it should be possible to obtain quite reliable assessments” (Adler et al., p. 236). Even the con side noted, “In theory, it is possible to obtain ratings from multiple raters and pool them to eliminate some types of interrater agreement” (Adler et al., p. 225), although this side was certainly less optimistic about the merits of multiple raters. In the broader industrial–organizational psychology literature, authors have repeatedly heralded the benefits of adding additional raters for performance ratings, some even treating it as a panacea for inaccurate ratings. Although these authors extol the virtues of multiple raters, an important question is often omitted from relevant discussions of performance ratings: To what extent do additional raters actually improve performance ratings? Does adding an additional rater double the validity of performance ratings? Does an additional rater increase the validity of performance ratings by a constant value? Or is the answer something else altogether?

Although we share Bergman and Jean's (2016) concerns about the representativeness of samples in the organizational sciences, we are mindful of the ever changing nature of the job market. New jobs are created from technological innovation while others become obsolete and disappear or are functionally transformed. These shifts in employment patterns produce both opportunities and challenges for organizational researchers addressing the problem of the representativeness in our working population samples. On one hand, it is understood that whatever we do, we will always be playing catch-up with the market. On the other hand, it is possible that we can leverage new technologies in order to react to such changes more quickly. As an example, in Bergman and Jean's commentary, they suggested making use of crowdsourcing websites or Internet panels in order to gain access to undersampled populations. Although we agree there is an opportunity to conduct much research of interest to organizational scholars in these settings, we also would point out that these types of samples come with their own sampling challenges. To illustrate these challenges, we examine sampling issues for Amazon's Mechanical Turk (MTurk), which is currently the most used portal for psychologists and organizational scholars collecting human subjects data online. Specifically, we examine whether MTurk workers are “workers” as defined by Bergman and Jean, whether MTurk samples are WEIRD (Western, educated, industrialized, rich, and democratic; Henrich, Heine, & Norenzayan, 2010), and how researchers may creatively utilize the sample characteristics.

And the beat goes on. The same questions about performance appraisals keep popping up despite significant changes in work environments, contexts, and expectations over the past 2 or 3 decades (Adler et al., 2016). Even after decades of research and debate about the benefits and construction of performance appraisal ratings, no closure is reached or “best practice” identified. The application of ratings differs widely among companies, and the criteria, scaling, and language are tweaked by virtually every human resources group. In my experience, each organization believes that its performance criteria are unique. This should not be surprising because supervisors who observe and rate human performance do not react like a school of fish. What most human resources managers miss is that each of the supervisors who apply ratings are also unique, and they do not perceive performance consistently—except, perhaps, for the most exceptional and the poorest performers. Methods of quantifying or behaviorally slotting employee performance along a variety of dimensions to arrive at some accurate scaled rating have not made employees happy and are a painful chore for most supervisors.

We concur with Bergman and Jean (2016) that worker samples tend to be underrepresented in organizational research, which could have deleterious effects on the conclusions and practices derived from this research. However, we argue that the effects of underrepresentation could vary considerably by organizational domain (e.g., selection vs. leadership vs. satisfaction). Our focus with this commentary is to address its effects on organizational selection, particularly with employment interviews, including issues such as criterion-related validity, prevalence of student samples, and the four criteria outlined by the focal authors (overlooked phenomena, differences in construct meaning, worker status, and human capital patterns).

Adler et al. (2016) present a comprehensive review of the fundamental and painful problems of performance appraisal in organizations, from which it appears that we cannot do without performance appraisal because of its benefits, and we cannot do with it because of its flaws. If so, how can organizations make a decision on this issue? Mend them or end them? (See also Culbertson, Krome, McHenry, Stetzer, & van Ittersum, 2013.)

Bergman and Jean (2016) rightly argue that published research in industrial–organizational (I-O) psychology often underrepresents low-wage and frontline employees in favor of professional workers and management. One possible consequence of this bias is that I-O research may unintentionally marginalize workplace phenomena that impact employees professionally and personally. One example offered by Bergman and Jean is economic tenuousness, a work–life stressor that is more likely to be experienced by low-income and frontline employees. The recent growth in the proportion of individuals employed in low-wage jobs (Albelda & Carr, 2012) reinforces the need to explore the impact of the publication rift between the science and practice of I-O psychology.

Past research purporting to study employee resilience suffers from a lack of conceptual clarity about both the resilience construct and the methodological designs that examine resilience without ensuring the occurrence of significant adversity. The overall goal of this article is to address our contemporary understanding of employee resilience and identify pathways for the future advancement of resilience research in the workplace. We first address conceptual definitions of resilience both inside and outside of industrial and organizational psychology and make the case that researchers have generally failed to document the experience of significant adversity when studying resilience in working populations. Next, we discuss methods used to examine resilience, with an emphasis on distinguishing the capacity for resilience and the demonstration of resilience. Representative research is then reviewed by examining self-reports of resilience or resilience-related traits along with research on resilient and nonresilient trajectories following significant adversity. We then briefly address the issues involved in selecting resilient employees and building resilience in employees. The article concludes with recommendations for future research studying resilience in the workplace, including documenting significant adversity among employees, assessing multiple outcomes, using longitudinal designs with theoretically supported time lags, broadening the study of resilience to people in occupations outside the military who may face significant adversity, and addressing the potential dark side of an emphasis on resilience.

The focal article “How Much Do We Really Know About Employee Resilience?” by Britt, Shen, Sinclair, Grossman, and Klieger (2016) is an informative work concerning the paucity of resilience research in organizational science. This commentary serves two purposes: (a) to provide specific recommendations relevant to the aims of the focal article and (b) to present additional unexplored areas that would contribute to what we “really know about employee resilience.” First, I offer suggestions concerning two fundamental issues within resilience research mentioned by Britt et al.: the lack of conceptual clarity and the lack of agreement concerning the characterization of significant adversity. Second, I present three directions for future research: incorporating levels of analysis, developing resilience typologies, and deepening theoretical representations. This commentary complements the focal article by highlighting both specific recommendations and novel areas deserving of exploration.

The purpose of our commentary is to expand the conversation started by the focal article (Bergman & Jean, 2016) by providing a practitioner perspective, especially focusing on practitioners who work in a government or military context. We believe this perspective is valuable because (a) interventions and research projects involving nonprofessional/nonmanagement workers are already regularly being conducted in this context; (b) there are likely to be other causes of a lack of focus on broader populations of workers in this context than those highlighted by the focal article, some of which may differ between practitioners and academics; and (c) we believe practitioners in this context have much to contribute to solving the potential problems that the focal article highlights.

According to the focal article by Britt, Shen, Sinclair, Grossman, and Klieger (2016), it seems conclusive that all definitions of resilience involve an experience of significant adversity, regardless of whether it is examined as a trait or an outcome. This experience of adversity is inherently emotional. When considering the ability or outcome of “bouncing back” from a stressful or chronic event, one must recognize the emotional experience and consider how individuals may cope with their emotions. This said, there is a clear connection between resilience and emotion regulation. The focal article presents a descriptive model of resilience for employees, which includes mention of energy and affect as individual resources but does not acknowledge the connection between resilience and emotion regulation. In this commentary I argue that these two research areas are related but largely neglected in the current literature. I will discuss the (a) process model of emotion regulation, (b) points of connection with resilience, and (c) empirical research suggesting the importance of positive emotion.

Bergman and Jean (2016) suggest that an underrepresentation of nonmanagerial workers in industrial and organizational (I-O) research is hindering our contribution to practice and ultimately our legitimacy as a discipline. I agree with the target article authors’ overarching message insomuch as it serves as a reminder that we, as I-O psychologists, have responsibilities to understand and improve the work experience of a diverse range of populations. However, I also believe the authors’ claim that nonmanagerial workers are underrepresented may be overstated. Moreover, I offer a perspective suggesting that the current supply of research being produced in the name of I-O psychology is driven by real world demand, that this naturally occurring phenomenon is what maximizes our impact on practice, and that ignoring this phenomenon is actually what hinders the legitimacy of applied disciplines.

Bergman and Jean (2016) skillfully summarize how the industrial and organizational (I-O) psychology literature generally overrepresents salaried, core, managerial, professional, and executive employees. We concur that that the underrepresentation of traditional workers (i.e., wage earners, laborers, first-line personnel, freelancers, contract workers, and other workers outside managerial, professional, and executive positions) can negatively affect our science. In our commentary we extend the arguments of Bergman and Jean by (a) discussing the appropriate use of samples, which are determined by study goals and hypotheses, and (b) further examining samples in occupational health psychology (OHP) and related journals, which generally require worker samples.

Understanding resilience is important to creating and maintaining health in the workplace, and the focal article by Britt, Shen, Sinclair, Grossman, and Klieger (2016) raises valuable questions and recommendations for research in the field. In this commentary we consider several issues not discussed by Britt et al. but critical to understanding resilience in organizational settings. In particular, we discuss the utility of process-oriented models and, specifically, the role of self-regulatory processes as foundational mechanisms of resiliency. We agree with many of Britt et al.’s recommendations and provide additional perspectives and information based on recent research on resiliency in military personnel experiencing cross-cultural adversity, in executives experiencing unwanted career transitions, and in recent immigrants searching for employment.

Since its inception in mainstream scientific psychology, resilience has emerged as a popular, yet controversial, construct. As evidenced in the target article, this controversy can be attributed, at least in part, to current conceptual and methodological difficulties hampering understanding of the construct. Chief among these concerns is the discriminant validity of resilience with respect to conceptually similar individual differences constructs, such as hardiness, mental toughness, adaptability, and even the five-factor personality dimensions. Advances in research on resilience, and ultimately the utility of the construct in applied settings, hinge on its distinction from related concepts.