Introduction
Depressive symptoms have a high prevalence and can be accompanied by many mental illnesses, including schizophrenia, bipolar disorder, and major depressive disorder. The lifetime risk of experiencing depressive disorders is estimated to be between 15% and 18%, and the probability of experiencing depressive symptoms is even greater (Malhi & Mann, Reference Malhi and Mann2018; Shorey, Ng, & Wong, Reference Shorey, Ng and Wong2022; Substance Abuse and Mental Health Services Administration, 2021). Depressive symptoms are correlated with adverse clinical outcomes, including hopelessness, suicidal behavior, and demoralization, highlighting the critical need for early identification and treatment of these symptoms (Berardelli et al., Reference Berardelli, Serafini, Cortese, Fiasche, O'Connor and Pompili2020; Costanza et al., Reference Costanza, Vasileios, Ambrosetti, Shah, Amerio, Aguglia and Berardelli2022). Furthermore, in patients with other mental disorders not specifically diagnosed as depressive disorders, the presence of depressive symptoms can lead to greater functional impairment (Berardelli et al., Reference Berardelli, Rogante, Sarubbi, Erbuto, Lester and Pompili2021b; Upthegrove, Marwaha, & Birchwood, Reference Upthegrove, Marwaha and Birchwood2017). Children and adolescents are particularly vulnerable to depressive symptoms, and the prevalence of these symptoms in children and adolescents is reportedly greater than that in adults (Miller & Campo, Reference Miller and Campo2021). A significant number of adults who experience depressive symptoms trace their symptoms back to adolescence, and those with symptoms that emerge in adolescence often have poorer prognoses (Berardelli et al., Reference Berardelli, Innamorati, Sarubbi, Rogante, Erbuto, Lester and Pompili2021a; Berardelli et al., Reference Berardelli, Sarubbi, Rogante, Erbuto, Cifrodelli, Giuliani and Pompili2022; Weavers et al., Reference Weavers, Heron, Thapar, Stephens, Lennon, Bevan Jones and Rice2021). Therefore, the early identification and treatment of depressive symptoms, especially in younger populations, is critically important (Cuijpers, van Straten, Smit, Mihalopoulos, & Beekman, Reference Cuijpers, van Straten, Smit, Mihalopoulos and Beekman2008; Mendelson & Tandon, Reference Mendelson and Tandon2016).
Digital phenotyping is the process of measuring and analyzing individuals' behavioral and physiological characteristics using digital tools and technologies. This process typically involves the use of mobile devices such as smartphones to collect information on users' smartphone usage patterns, location data, social media activity, keyboard usage, and other sensor data. The collected data are subsequently utilized to monitor behaviors and diseases (Huckvale, Venkatesh, & Christensen, Reference Huckvale, Venkatesh and Christensen2019). Digital phenotyping is gaining significant attention as a valuable tool in mental health. Subtle changes can be detected early through digital phenotyping, allowing these minor alterations to be continuously monitored. Moreover, an advantage of this method is that it reflects individual characteristics, making it a powerful approach for personalized health monitoring and assessment (Brietzke et al., Reference Brietzke, Hawken, Idzikowski, Pong, Kennedy and Soares2019; Kamath, Leon Barriera, Jain, Keisari, & Wang, Reference Kamath, Leon Barriera, Jain, Keisari and Wang2022; Sequeira et al., Reference Sequeira, Perrotta, LaGrassa, Merikangas, Kreindler, Kundur and Strauss2020; Torous et al., Reference Torous, Bucci, Bell, Kessing, Faurholt-Jepsen, Whelan and Firth2021).
In digital phenotyping, two primary types of data are utilized. Active data are consciously provided or input by users based on their subjective perceptions and experiences. However, potentially low compliance is a disadvantage of utilizing this type of data because it requires active participation from users, and its accuracy can vary based on users' memories and perceptions. In contrast, passive data are automatically collected without the user's conscious involvement and include smartphone sensor data, location tracking data, and online activity logs. Devices or applications can automatically collect these passive data without direct user participation. They are not influenced by users' subjective evaluations or inputs and therefore reflect objective situations or activities. Additionally, passive data are collected continuously, yielding a real-time data stream (Abbas et al., Reference Abbas, Sauder, Yadav, Koesmahargyo, Aghjayan, Marecki and Galatzer-Levy2021; Torous et al., Reference Torous, Bucci, Bell, Kessing, Faurholt-Jepsen, Whelan and Firth2021). Due to these advantages, our research focused on exploring and utilizing passive data.
There are numerous types of passive data, including smartphone usage data, location data, biometric signals, environmental data, and keyboard and mouse usage patterns (Mohr, Zhang, & Schueller, Reference Mohr, Zhang and Schueller2017). Among these types of data, we were especially interested in keyboard and stylus data. Keyboard data allow a user's emotional state and cognitive functions to be inferred by analyzing keystroke speed, length, and duration. Stylus data offer insights into fine motor skills and emotional states by enabling the analysis of hand movements and pressure during stylus use as well as usage patterns (Mastoras et al., Reference Mastoras, Iakovakis, Hadjidimitriou, Charisis, Kassie, Alsaadi and Hadjileontiadis2019; Vesel et al., Reference Vesel, Rashidisabet, Zulueta, Stange, Duffecy, Hussain and Leow2020). While other types of passive data, such as location and app usage data, can reflect a user's behavioral patterns and activity levels, they are less capable of capturing the subtle cognitive and emotional states that can be captured using keyboard or stylus data. Given these characteristics, multiple reports suggest that keyboard and stylus data have the potential to reflect depressive symptoms in real time and capture fine distinctions, highlighting their utility in monitoring nuanced mental health states (Brietzke et al., Reference Brietzke, Hawken, Idzikowski, Pong, Kennedy and Soares2019; Kamath et al., Reference Kamath, Leon Barriera, Jain, Keisari and Wang2022; Sequeira et al., Reference Sequeira, Perrotta, LaGrassa, Merikangas, Kreindler, Kundur and Strauss2020; Torous et al., Reference Torous, Bucci, Bell, Kessing, Faurholt-Jepsen, Whelan and Firth2021).
Previous studies have explored the correlation between keystroke metadata and depression (Bennett, Ross, Baek, Kim, & Leow, Reference Bennett, Ross, Baek, Kim and Leow2022; Cao et al., Reference Cao, Zheng, Zhang, Yu, Piscitello, Zulueta and Leow2017; Huang, Cao, Yu, Wang, & Leow, Reference Huang, Cao, Yu, Wang and Leow2018; Mastoras et al., Reference Mastoras, Iakovakis, Hadjidimitriou, Charisis, Kassie, Alsaadi and Hadjileontiadis2019; Stange et al., Reference Stange, Zulueta, Langenecker, Ryan, Piscitello, Duffecy and Leow2018; Vesel et al., Reference Vesel, Rashidisabet, Zulueta, Stange, Duffecy, Hussain and Leow2020; Zulueta et al., Reference Zulueta, Piscitello, Rasic, Easter, Babu, Langenecker and Leow2018). However, few studies have evaluated the specific relationships between the individual components of keystroke data and depressive symptoms (Bennett et al., Reference Bennett, Ross, Baek, Kim and Leow2022; Mastoras et al., Reference Mastoras, Iakovakis, Hadjidimitriou, Charisis, Kassie, Alsaadi and Hadjileontiadis2019). Moreover, only one paper has investigated the correlation between depressive symptoms and keystroke metadata in children and adolescents (Braund et al., Reference Braund, O'Dea, Bal, Maston, Larsen, Werner-Seidler and Christensen2023). This paper demonstrated a positive correlation between mental symptoms, such as depression, anxiety, distress, and insomnia, and typing speed. However, no studies that specifically targeted adolescents were found. Based on these previous studies, we hypothesized that keystroke and stylus metadata would be correlated with depressive symptoms in adolescents. In particular, this study aimed to examine how specific components of keystroke and stylus metadata correlate with depressive symptoms in adolescents.
Methods
Participants
The study involved 927 first-year middle school students with an average age of 13 years. Of these participants, 483 (52.2%) were female. In Seoul, South Korea, there are 281 middle schools with a total of 66,358 first-year middle school students. To identify individuals who were willing to participate in the research, official letters were sent to each school to solicit individual applications. Enrollment in the study was contingent upon voluntary consent to participate from both the students and their parents. Consent forms were obtained from 935 students and their parents simultaneously. Individuals with intellectual disabilities or autism spectrum disorders, individuals with central nervous system conditions such as epilepsy or significant head injuries, individuals experiencing severe health problems that might result in psychological symptoms, and individuals who were unable to use study-specific tablets were excluded. Before participation, students and their parents or legal guardians were informed of and consented to the study's purpose, methodology, and type of data to be collected. Information on the participants is summarized in Table 1.
Table 1. Demographic and clinical characteristics of the participants
Clinical and digital data collection
On the day of data collection, the students arrived at Seoul National University in Seoul, South Korea. The students were randomly assigned to classrooms in groups of approximately 30, replicating their regular classroom study environment. The desks and chairs were set up like those used in a typical school setting, with randomly assigned seating. A 30 minute session before data collection was provided to familiarize the students with the system interface and check for technical errors. Subsequently, the students completed the Patient Health Questionnaire-9 (PHQ-9), a widely used screening tool that is highly effective in screening for depressive symptoms among adolescents (Richardson et al., Reference Richardson, McCauley, Grossman, McCarty, Richards, Russo and Katon2010).
The students then spent 45 minute solving 20 Korean language problems aligned with the first-year middle school curriculum. All questions were subjective and required the students to type their responses using a keyboard. After a 15 minute break, the students spent another 45 minute solving 20 mathematics problems suited to their grade level. These math questions were subjective and were answered using a digital pen. During the Korean language and mathematics problem-solving sessions, the students were allowed to freely use both a keyboard and a digital stylus. While there was an overall time limit, there were no time constraints for individual questions. The students could also perform actions such as underlining or taking notes alongside the problems. An example of the screen on which the students solved the problems is shown in Fig. 1.
Figure 1. Examples of problems solved by students during the process of digital data collection. Although the problems and options were presented in Korean, they have been translated to English for this paper. (a) An example of a Korean language problem for which responses were typed. (b) Example of a math problem for which responses were provided using a stylus pen.
Digital data were collected using two passive sensors built into the tablet computers while the students worked on the tasks. These sensors included an on-screen keyboard sensor and a touchscreen sensor. The tablets used were Samsung Galaxy Tab S7 FE models with Android 11, the same as those used by the students for their regular schoolwork. The students positioned their tablets on the desktop to perform typing and keyboard activities, intentionally reducing any possible sounds from the movement of the devices. The data were recorded unobtrusively at 24 Hz (or every 42.67 milliseconds) during the learning activities via a preinstalled module called Dr. Simon on the tablets. This process captured 77 distinct types of interactions with the keyboard and stylus, such as keystroke count, speed, acceleration, length, duration, trajectory, and pressure. The missing data accounted for 0.9% of the variance and were attributable to device or network errors.
In accordance with the guidelines set forth in the Declaration of Helsinki, the students and their parents were thoroughly informed about the study, after which written informed consent was obtained. The study's methodology was formally approved by the Institutional Review Board of Korea University, as indicated by authorization number IRB-2023-0144. All procedures were carried out following review and approval by the Seoul Metropolitan Office of Education to ensure ethical compliance.
Statistical analysis
In this study, we standardized the measurements of various indicators, including self-reported clinical scores, frequencies, durations (measured in milliseconds), lengths (measured in pixels), pressure intensities, and ratios. This was achieved through z-score normalization by setting the mean of each indicator to zero and the standard deviation to one. This normalization allowed for comparisons across indicators with different distribution characteristics.
Traditionally, multiple regression with stepwise selection is a popular technique for identifying crucial indicators or measurements. Instead, due to reliability and adequacy issues, we applied a bootstrap-based multiple regression approach. For this purpose, we generated 10,000 sample sets using the Mersenne Twister algorithm for random number generation. This approach enhanced the stability of the model and was useful for handling outliers.
Results
Multiple regression
Regression analysis of the PHQ-9 scores was performed. In this model, the average keystroke acceleration, the number of incorrect answers, the average keystroke length, the average tap pressure, and the average keystroke speed were found to be predictors of depressive symptom scores. The results indicated that a higher average keystroke acceleration and length as well as a greater number of incorrect answers were associated with higher PHQ-9 scores. Conversely, a lower tap pressure and stroke speed were associated with higher PHQ-9 scores. The R 2 value was 0.0252. The results of this linear regression are presented in Fig. 2. Additionally, a dot plot illustrating the correlation between the standardized variables with a focus on the highest coefficient observed between the average stroke length and PHQ-9 score is displayed in Fig. 3.
Figure 2. Heatmap showing the associations between PHQ-9 scores and keystroke features.
Figure 3. Association between the PHQ-9 score and average stroke acceleration.
Discussion
This study represents a pioneering effort to reveal the digital phenotype of depressive symptoms among adolescents. The ultimate goal of the study was to enable the early diagnosis and timely intervention of mental health issues among students within the school setting. In this study, the digital data variables that explained the PHQ-9 scores included the number of incorrect answers; the length, speed, and acceleration of strokes; the average vertical position of strokes; and the average pressure exerted while typing on the keyboard. Higher PHQ-9 scores were associated with a lower tap pressure, a greater number of incorrect answers, and more frequent use of strokes toward the upper part of the tablet. Additionally, PHQ-9 scores decreased as the average stroke length increased, the average stroke speed decreased, and the average keystroke acceleration increased.
The regression analysis results indicated that greater tap pressure was associated with lower PHQ-9 scores. A lack of energy and lethargy are typical symptoms of depression. We hypothesized that these depressive symptoms might influence tap pressure. Moreover, studies published in 2018 and 2014 reported that adults exhibit greater keyboard pressure under high stress (Exposito, Hernandez, & Picard, Reference Exposito, Hernandez and Picard2018; Hernandez, Paredes, Roseway, & Czerwinski, Reference Hernandez, Paredes, Roseway and Czerwinski2014). These studies suggested that the observed phenomenon could be due to increased muscle tension during stressful situations. While stress and depressive states are distinct, the correlation between mental symptoms and tap pressure, mediated by muscle tone, is noteworthy. Depressive symptoms can manifest as either psychomotor agitation or psychomotor retardation. In adolescents, psychomotor retardation is more commonly observed in the presence of depressive symptoms, which could contribute to the negative correlation between tap pressure and depressive symptoms observed in this study. However, the correlation between tap pressure and depressive symptoms is not well documented, suggesting the need for more extensive research in this area.
The regression analysis results revealed that higher PHQ-9 scores were associated with a greater number of incorrect answers and a greater distribution of strokes toward the upper part of the tablet. Depression is closely linked to impaired cognitive functions such as concentration and is known to cause functional impairments. Notably, a decline in academic performance among students is a well-documented issue associated with depression (Mangione et al., Reference Mangione, Barry, Nicholson, Cabana, Chelmow, Coker and Wong2022). Thus, the positive correlation between the number of incorrect answers and the depression score is consistent with established knowledge. However, the correlation between strokes occurring predominantly in the upper part of the tablet and symptoms of depression is more challenging to explain. Considering that stroke data were collected mainly during math problem-solving sessions, where problems are typically positioned at the top and answers are typically positioned at the bottom, it is plausible that individuals might spend more time thinking about problems than writing answers, resulting in strokes being predominantly located in the upper part of the tablet. This could be related to depression given its close association with cognitive impairments such as reduced concentration. Nevertheless, this remains a hypothesis, and more detailed data collection and analysis are required for validation.
This study revealed that more depressive symptoms were associated with longer stroke duration, slower stroke speed, and faster stroke acceleration. Previous research focusing on writing patterns in patients with depression, particularly individuals aged 30–40 years, revealed that patients with depressive symptoms typically exhibit slower stroke speeds with greater variation and longer durations and lengths of strokes (Mergl et al., Reference Mergl, Juckel, Rihl, Henkel, Karner, Tigges and Hegerl2004, Reference Mergl, Pogarell, Juckel, Rihl, Henkel, Frodl and Hegerl2007). The authors of these studies argued that these observations could be related to the psychomotor retardation and basal ganglia dysfunction observed in patients with depression. These findings are consistent with the results of the present study, in which longer stroke lengths and slower stroke speeds were observed. Additionally, the high variability in stroke speed may suggest greater stroke acceleration. Therefore, the results of this study align with previous research findings. Furthermore, a previous study reported a correlation between baseline typing instability and depressive symptoms, which also corresponds with the positive correlation between depressive symptoms and acceleration observed in the present study (Stange et al., Reference Stange, Zulueta, Langenecker, Ryan, Piscitello, Duffecy and Leow2018).
In previous research conducted with children as participants, depressive symptoms were reported to be associated with faster typing speeds (Braund et al., Reference Braund, O'Dea, Bal, Maston, Larsen, Werner-Seidler and Christensen2023). The authors of this study attributed the observed phenomenon to psychomotor agitation and noted that other research predominantly showed that slower typing speeds could be due to psychomotor retardation. Consequently, the authors advocated for additional research to further explore these disparities. The variations are likely due to the different manifestations of depressive symptoms in children, adolescents, and elderly people. In children, depressive symptoms often present as prominent psychomotor agitation, whereas in adolescents, psychomotor retardation is more pronounced (Sadock, Sadock, & Ruiz, Reference Sadock, Sadock and Ruiz2017). An earlier study in which data were collected from participants with an average age of 8 years revealed results indicative of psychomotor agitation. In contrast, the present study, which targeted adolescents, revealed findings that reflect psychomotor retardation.
The research findings highlight a significant relationship between higher depression scale scores and noticeable changes in keystroke and stylus data in contrast to lower depression scale scores. These differences could be attributed to psychomotor disruptions and cognitive impairment. The primary aim of this study was to pinpoint potential digital indicators that represent approximately four (depressive/irritable mood, psychomotor retardation/agitation, loss of energy/fatigue, loss of concentration) out of the nine symptoms (depressive/irritable mood, diminished interest/pleasure, significant increase/decrease in weight/appetite, insomnia/hypersomnia, psychomotor retardation/agitation, loss of energy/fatigue, feelings of excessive guilt, loss of concentration, suicidal ideation) of depression by analyzing keystroke and stylus data obtained in an examination context (Thapar, Eyre, Patel, & Brent, Reference Thapar, Eyre, Patel and Brent2022). In addition, children and adolescents frequently exhibit depressive symptoms without a concurrent sense of sadness and often overlook or inadequately report their symptoms. This study investigated the association between passively collected digital data in a routine academic testing environment and the manifestation of depressive symptoms. The cumulative efforts of this research, which aimed to establish a digital phenotype for depression in adolescents, could empower educators to identify and address depressive symptoms in students in a timely manner.
Strengths and limitations
This study has several limitations. First, only keystroke and stylus data were collected, and other passive data were not obtained. Additionally, the data were collected in less than 3 hours, making it challenging to adequately capture longitudinal changes. These factors may have contributed to the limited explanatory power observed in the multiple regression analysis. The fact that clinicians did not diagnose the participants directly but relied on self-reported scores to measure depressive symptoms is also a limitation. Furthermore, the use of the PHQ-9 instead of the PHQ-9 modified for Adolescents (PHQ-A) may also be a limitation. Additionally, the sample used in this study included almost no adolescents with clinically significant psychopathology, which may be considered a limitation. Above all, due to the scarcity of research identifying digital phenotypes associated with depression in adolescents, this study assumes a preliminary nature, making it challenging to design and test precise hypotheses based on existing research.
This study aimed to replicate a school examination scenario to passively collect data. This could serve as a foundation for establishing a system within schools where teachers can regularly screen students for depressive symptoms without additional effort and allow for prompt intervention when necessary. Another notable strength of this research compared to previous digital phenotype studies of depression is the considerably large sample size. Furthermore, ethical considerations concerning the capacity for informed consent in digital phenotype studies involving adolescents are scarce. This study's strength lies in its focus on 13-year-old adolescents.
Conclusions
This research can be considered a foundational study in the emerging field of digital phenotype research rather than a conclusive study, particularly for understanding depressive symptoms in adolescents. Building upon this foundation, more rigorous hypotheses and experimental designs must be formulated for further research. Based on this research, by conducting a sufficiently long longitudinal study to collect digital data over an extended period and performing subsequent research including a long-term and meticulous evaluation of clinically significant pathology, it would be plausible to screen for and promptly intervene in depressive symptoms in adolescents, which would lead to significant societal costs if they become chronic. Additionally, research that focuses on identifying digital phenotypes capable of predicting the development of various mental disorders, including suicide risk and schizophrenia, in students will not only facilitate early diagnosis and timely intervention but also ultimately aid in reducing community health expenses.
Acknowledgments
The authors extend their deep gratitude to the Seoul Metropolitan Office of Education for their outstanding support and collaboration. As the Office spearheaded the Digital Buddy (Di-Bud) project aimed at digitally transforming schools, their assistance was invaluable in gathering essential digital data using the Dr. Simon (3R Innovation Inc. URL: https://drsimon.ai). This research was supported by the Bio & Medical Technology Development Program, the Brain Science Convergence Research Program, and an Institute of Information & Communications Technology Planning & Evaluation (IITP) grant (2021M3A9E408078412, RS-2023-00266120, and RS-2021-II212068), which is funded by the Ministry of Science & ICT. This research was also supported by the Alchemist Project (1415187369, 20025696), which is funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).
Open access
