So, where is unproctored Internet testing (UIT) now? What has happened since 2006? Is the train stalled right outside the station, waiting on answers? Is it lurching forward, making slow but sure progress? Or, is it simply chugging in circles? The original focal article (Tippins, 2009) and the 13 subsequent commentaries are not comprehensive summaries of all the research and thought about UIT and can hardly be considered definitive as a collective body of work. In total, these articles probably do not convince many of the doubters. However, the articles collectively highlight the critical issues faced in UIT, suggest new ideas, and give guidance for the future, indicating the UIT train is at least moving. In this concluding article of the UIT series, I try to summarize where I believe we are in 2009 in terms of the important points that must be taken into consideration when contemplating UIT and emerging questions that industrial and organizational (I–O) psychologists must answer.

In the words of Winston Churchill, “When you're going through hell, keep going.” Britt, Shen, Sinclair, Grossman, and Klieger (2016) note several traits (e.g., individual resources), environmental factors (e.g., unit, family, and community resources), and processes (e.g., seeking help from others) that help individuals to “keep going” in the face of adversity. I would argue that the third category, which I would suggest be expanded to practices, is the most important going forward. Unfortunately, psychologists often tend to focus the most effort on the first two: traits and environmental factors, which often leave individuals feeling helpless because both are largely outside of their control. In the face of adversity, people instead want to know, “What can I do to keep going?” This is not to say that traits or environmental factors are not important. They are. However, the most powerful work in resilience will promote personal agency (Bandura, 2001) and confidence in one's ability to develop resilience (e.g., a growth mindset; Dweck, 2006).

As Gardner, Ryan, and Snoeyink (2018) state, their findings on gender representation in industrial and organizational (I-O) psychology indicate that “the profession as a whole falls into the category of ‘not walking the talk’” (p. 385). We agree that it is imperative to understand the current state of gender inequity in our field while also actively working toward achieving gender equity. This article attempts to inspire each and every individual in I-O psychology to feel a personal responsibility to engage in behaviors that reduce gender disparities in our field. Although women are normatively the focus in fights for gender equity, men should be equal partners in these efforts. In this commentary, we focus on the contributions that male allies in I-O psychology can make in fostering gender equity. To be clear, we are not claiming that women need to be rescued by men; however, we do believe that I-O psychology can achieve the greatest progress toward gender equity when both women and men engage in supportive efforts. As Emma Watson said in her 2014 United Nations speech, “How can we affect change in the world when only half of it is invited or feel welcome to participate in the conversation?” (UN Women, 2014). In times when political leaders and national laws may fail women, it is crucial that local communities—like the I-O community—adopt a clear stance in promoting gender equity. In this commentary, we define allyship, discuss the importance of male allies, suggest ways in which male allies can help promote gender equity in I-O psychology, and consider potential barriers to male allyship and ways to overcome them. The strategies that we propose are by no means exhaustive; rather, they are suggestions for how to initiate a larger movement.

Since its inception in mainstream scientific psychology, resilience has emerged as a popular, yet controversial, construct. As evidenced in the target article, this controversy can be attributed, at least in part, to current conceptual and methodological difficulties hampering understanding of the construct. Chief among these concerns is the discriminant validity of resilience with respect to conceptually similar individual differences constructs, such as hardiness, mental toughness, adaptability, and even the five-factor personality dimensions. Advances in research on resilience, and ultimately the utility of the construct in applied settings, hinge on its distinction from related concepts.

Landers and Behrend (2015) make a good argument that more consideration should be given to sampling strategies in light of the specific research question prior to data collection and that nonorganizational samples should not be automatically dismissed by journal editors and reviewers. Yet, the authors only briefly mention one particular issue that is also relevant to the validity of our research findings—participant motivation. Researchers should seek to better understand why individuals choose to participate in a study and what may be motivating the levels of effort they put forth in participating. Two critical questions include Are participants who they say they are (e.g., working adults)? And, are participants paying attention to the study instructions and questions and participating with effort? In this response, I expand on issues related to participant motivation and apply them to the sampling strategies discussed by Landers and Behrend (2015). I also provide suggestions for ways researchers may address these issues.

In industrial-organizational (I-O) psychology, much like in the organizational sciences more broadly (Hambrick, 2007), we have a bit of an addiction to theoretical models. It is commonly assumed that developing new theory is the most valuable way to solve pressing research problems and to drive our field forward (Mathieu, 2016). However, this assumption is untested, and there is growing awareness among organizational scientists that this hardline approach, which is unusual among both the natural sciences and other social sciences, may even be damaging the reputation and influence of our field (Antonakis, 2017; Ones, Kaiser, Chamorro-Premuzic, & Svensson, 2017). As Hambrick (2007) describes, the requirement for theory first “takes an array of subtle, but significant, tolls on our field” (p. 1348). As we will describe in this article, Morelli, Potosky, Arthur, and Tippins’ (2017) suggestions, if taken at face value, will likely create such tolls by encouraging the creation of new theories of dubious value. To be clear, we agree with Morelli et al. that better theory is needed for technology's impact on I-O psychology broadly and talent assessment in particular. We disagree, however, that creating new technology theories using the approaches that I-O psychology typically employs is likely to accomplish this broader goal. Rather, it will ultimately only isolate research on I-O technologies even further from both mainstream I-O research and technology research. Given that we are already quite isolated, this would be a disastrous path.

Although Lievens and Motowidlo (2016) made a strong case for reconceptualizing situational judgment tests (SJTs) as measures of general domain knowledge, we disagree with their view that the judgment or assessment of the situation itself is not important. We contend that situation assessment is an integral yet ignored factor in SJTs and that both general domain knowledge and situation assessment are needed to better understand how SJTs work.