Machine learning of language use on Twitter reveals weak and non-specific predictions

Via Peters
  • Abuse, S. Mental Health Services Administration. Key substance use and mental health indicators in the United States: results from the 2018 National Survey on Drug Use and Health (HHS Publication No. PEP19-5068, NSDUH Series H-54) (Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration, Rockville, MD, 2019).

  • Lépine, J.-P., Gastpar, M., Mendlewicz, J. & Tylee, A. Depression in the community: the first pan-European study DEPRES (Depression Research in European Society). Int. Clin. Psychopharmacol. 12, 19–29 (1997).

    PubMed 

    Google Scholar
     

  • Ghio, L., Gotelli, S., Marcenaro, M., Amore, M. & Natta, W. Duration of untreated illness and outcomes in unipolar depression: a systematic review and meta-analysis. J. Affect. Disord. 152, 45–51 (2014).

    PubMed 

    Google Scholar
     

  • Perrin, A. Social media usage. Pew Res. Cent. 125, 52–68 (2015).


    Google Scholar
     

  • De Choudhury, M., Gamon, M., Counts, S. & Horvitz, E. Predicting depression via social media. International AAAI Conference on Web and Social Media. 2, 128–137 (AAAI, 2013).

  • De Choudhury, M., Counts, S., Horvitz, E. J. & Hoff, A. Characterizing and Predicting Postpartum Depression from Shared Facebook Data. In Proc. 17th ACM Conference on Computer supported cooperative work & social computing (CSCW). 626–638 (ACM, 2014).

  • Tsugawa, S. et al. Recognizing depression from twitter activity. In Proc. ACM Conference on Human Factors in Computing Systems (CHI). 3187–3196 (ACM, 2015).

  • Eichstaedt, J. C. et al. Facebook language predicts depression in medical records. Proc. Natl Acad. Sci. USA 115, 11203–11208 (2018).

    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Reece, A. G. et al. Forecasting the onset and course of mental illness with Twitter data. Sci. Rep. 7, 13006 (2017).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Reece, A. G. & Danforth, C. M. Instagram photos reveal predictive markers of depression. EPJ Data Science 6. https://doi.org/10.1140/epjds/s13688-017-0110-z (2017).

  • De Choudhury, M. Anorexia on Tumblr : A Characterization Study on Anorexia. In Proc. 5th International Conference on Digital Health. 43–50 (ACM, 2015).

  • Wolf, M., Theis, F. & Kordy, H. Language use in eating disorder blogs. J. Lang. Soc. Psychol. 32, 212–226 (2013).


    Google Scholar
     

  • Wang, T., Brede, M., Ianni, A. & Mentzakis, E. Social interactions in online eating disorder communities: a network perspective. PLoS ONE 13, e0200800 (2018).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Chancellor, S., Lin, Z. J. J., Goodman, E. L., Zerwas, S. & De Choudhury, M. Quantifying and Predicting Mental Illness Severity in Online Pro-Eating Disorder Communities. In Proc. 19th ACM Conference of Computer Supported Cooperative Work (CSCW). 1169–1182 (ACM, 2016).

  • Mitchell, M., Hollingshead, K. & Coppersmith, G. Quantifying the language of schizophrenia in social media. In Proc. 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. 11–20(ACL, 2015).

  • Birnbaum, M. L., Ernala, S. K., Rizvi, A. F., De Choudhury, M. & Kane, J. M. A collaborative approach to identifying social media markers of schizophrenia by employing machine learning and clinical appraisals. J. Med. Internet Res. 19, e289 (2017).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • McManus, K., Mallory, E. K., Goldfeder, R. L., Haynes, W. A. & Tatum, J. D. Mining Twitter data to improve detection of schizophrenia. AMIA Summits Transl. Sci. Proc. 2015, 122 (2015).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Nobles, A. L., Glenn, J. J., Kowsari, K., Teachman, B. A. & Barnes, L. E. Identification of Imminent Suicide Risk Among Young Adults using Text Messages. Proc. SIGCHI Confer. Hum. Factor Comput. Syst. https://doi.org/10.1145/3173574.3173987 (2018).

  • Coppersmith, G., Ngo, K., Leary, R. & Wood, A. Exploratory analysis of social media prior to a suicide attempt. In Proc. 3rd Workshop on Computational Linguistics and Clinical Psychology. 106–117 (ACL, 2016).

  • Cheng, Q., Li, T. M., Kwok, C. L., Zhu, T. & Yip, P. S. Assessing suicide risk and emotional distress in Chinese Social Media: a text mining and machine learning study. J. Med. Internet Res. 19, e243 (2017).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Coppersmith, G., Leary, R., Crutchley, P. & Fine, A. Natural language processing of social media as screening for suicide risk. Biomed. Inf. Insights 10, 1178222618792860 (2018).


    Google Scholar
     

  • Rude, S., Gortner, E.-M. & Pennebaker, J. Language use of depressed and depression-vulnerable college students. Cognition Emot. 18, 1121–1133 (2004).


    Google Scholar
     

  • Zimmermann, J., Brockmeyer, T., Hunn, M., Schauenburg, H. & Wolf, M. First-person pronoun use in spoken language as a predictor of future depressive symptoms: preliminary evidence from a clinical sample of depressed patients. Clin. Psychol. Psychother. 24, 384–391 (2017).

    PubMed 

    Google Scholar
     

  • Zimmermann, J., Wolf, M., Bock, A., Peham, D. & Benecke, C. The way we refer to ourselves reflects how we relate to others: associations between first-person pronoun use and interpersonal problems. J. Res. Personal. 47, 218–225 (2013).


    Google Scholar
     

  • Molendijk, M. L. et al. Word use of outpatients with a personality disorder and concurrent or previous major depressive disorder. Behav. Res Ther. 48, 44–51 (2010).

    PubMed 

    Google Scholar
     

  • De Choudhury, M., Counts, S. & Horvitz, E. Social Media As a Measurement Tool of Depression in Populations. In Proc. 5th Annual ACM Web Science Conference (WebSci). 47–56 (ACM, 2013).

  • Kessler, R. C. & Magee, W. J. Childhood adversities and adult depression: basic patterns of association in a US national survey. Psychological Med. 23, 679–690 (1993).

    CAS 

    Google Scholar
     

  • Insel, T. et al. Research Domain Criteria (RDoC): Toward a New Classification Framework for Research on Mental Disorders. Am. J. Psychiatry. 167, 748–751 (2010).

    PubMed 

    Google Scholar
     

  • Coppersmith, G., Dredze, M., Harman, C., Holli and Hollingshead, K. From ADHD to SAD: Analyzing the language of mental health on Twitter through self-reported diagnoses. In Proc. 2nd Workshop on Computational Linguistics and Clinical Psychology. 1–10 (Association for Computational Linguistics, 2015).

  • Cohan, A. et al. SMHD: a large-scale resource for exploring online language usage for multiple mental health conditions. Preprint at https://arxiv.org/abs/1806.05258 (2018).

  • Lyons, M., Aksayli, N. D. & Brewer, G. Mental distress and language use: linguistic analysis of discussion forum posts. Computers Hum. Behav. 87, 207–211 (2018).


    Google Scholar
     

  • Coppersmith, G., Dredze, M. & Harman, C. Quantifying Mental Health Signals in Twitter. In Proc. Workshop on Computational Linguistics and Clinical Psychology. 51–60 (Association for Computational Linguistics, 2014).

  • Wolf, M., Sedway, J., Bulik, C. M. & Kordy, H. Linguistic analyses of natural written language: unobtrusive assessment of cognitive style in eating disorders. Int J. Eat. Disord. 40, 711–717 (2007).

    PubMed 

    Google Scholar
     

  • Ernala, S. K., Rizvi, A. F., Birnbaum, M. L., Kane, J. M. & De Choudhury, M. Linguistic markers indicating therapeutic outcomes of social media disclosures of schizophrenia. Proc. ACM Hum. Comput Interact. 1, 1–27 (2017).


    Google Scholar
     

  • Zomick, J., Levitan, S. I. & Serper, M. Linguistic analysis of schizophrenia in Reddit posts. In Proc. Sixth Workshop on Computational Linguistics and Clinical Psychology. 74–83 (ACL, 2019).

  • Chancellor, S. & De Choudhury, M. Methods in predictive techniques for mental health status on social media: a critical review. NPJ digital Med. 3, 1–11 (2020).


    Google Scholar
     

  • Ireland, M. & Iserman, M. Within and between-person differences in language used across anxiety support and neutral reddit communities. In Proc. Fifth Workshop on Computational Linguistics and Clinical Psychology. 182–193 (ACL, 2018).

  • Brundage, M. et al. The malicious use of artificial intelligence: forecasting, prevention, and mitigation. Preprint at https://arxiv.org/abs/1802.07228 (2018).

  • Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H. & Eichstaedt, J. C. Detecting depression and mental illness on social media: an integrative review. Curr. Opin. Behav. Sci. 18, 43–49 (2017).


    Google Scholar
     

  • Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A. & Daw, N. D. Characterizing a psychiatric symptom dimension related to deficits in goal-directed control. Elife 5, e11305 (2016).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Edwards, T. M. & Holtzman, N. S. A meta-analysis of correlations between depression and first person singular pronoun use. J. Res. Personal. 68, 63–68 (2017).


    Google Scholar
     

  • Curtis, B. et al. Can Twitter be used to predict county excessive alcohol consumption rates? PLoS ONE 13, e0194290 (2018).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Prieto, V. M., Matos, S., Alvarez, M., Cacheda, F. & Oliveira, J. L. Twitter: a good place to detect health conditions. PLoS ONE 9, e86191 (2014).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Nakamura, T., Kubo, K., Usuda, Y. & Aramaki, E. Defining patients with depressive disorder by using textual information. In Proc. 2014 AAAI Spring Symposium Series (AAAI, 2014).

  • Leis, A., Ronzano, F., Mayer, M. A., Furlong, L. I. & Sanz, F. Detecting Signs of Depression in Tweets in Spanish: Behavioral and Linguistic Analysis. J. Med Internet Res 21, e14199 (2019).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Chen, X., Sykora, M. D., Jackson, T. W. & Elayan, S. What about mood swings: Identifying depression on twitter with temporal measures of emotions. In Companion Proceedings of the The Web Conference. 1653–1660 (2018).

  • Grant, J. E. & Chamberlain, S. R. Sleepiness and impulsivity: findings in non-treatment seeking young adults. J. Behav. Addict. 7, 737–742 (2018).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Van Veen, M., Karsten, J. & Lancel, M. Poor sleep and its relation to impulsivity in patients with antisocial or borderline personality disorders. Behav. Med. 43, 218–226 (2017).

    PubMed 

    Google Scholar
     

  • Fineberg, S. et al. Self-reference in psychosis and depression: a language marker of illness. Psychol. Med. 46, 2605 (2016).

    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Bucci, W. & Freedman, N. The language of depression. Bull. Menninger Clin. 45, 334 (1981).

    CAS 
    PubMed 

    Google Scholar
     

  • Hswen, Y., Gopaluni, A., Brownstein, J. S. & Hawkins, J. B. Using Twitter to detect psychological characteristics of self-identified persons with autism spectrum disorder: a feasibility study. JMIR Mhealth Uhealth 7, e12264 (2019).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kessler, R. C., Chiu, W. T., Demler, O. & Walters, E. E. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch. Gen. Psychiatry 62, 617–627 (2005).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kessler, R. C. et al. Comorbidity of DSM–III–R major depressive disorder in the general population: results from the US National Comorbidity Survey. Br. J. Psychiatry 168, 17–30 (1996).


    Google Scholar
     

  • Boschloo, L. et al. The network structure of symptoms of the diagnostic and statistical manual of mental disorders. PLoS ONE 10, e0137621 (2015).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Borsboom, D., Cramer, A. O., Schmittmann, V. D., Epskamp, S. & Waldorp, L. J. The small world of psychopathology. PLoS ONE 6, e27407 (2011).

    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Zhang, R. The stress-buffering effect of self-disclosure on Facebook: an examination of stressful life events, social support, and mental health among college students. Comput Hum. Behav. 75, 527–537 (2017).


    Google Scholar
     

  • De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G. & Kumar, M. Discovering shifts to suicidal ideation from mental health content in social media. Proc. ACM Conference on Human Factors in Computing Systems (CHI). 2016, 2098–2110 (ACM, 2016).

  • De Choudhury, M. & Kiciman, E. The language of social support in social media and its effect on suicidal ideation risk. In Proc. International AAAI Conference on Web and Social Media (AAAI, 2017).

  • Wojcik, S. & Hughes, A. Sizing up Twitter Users (Pew Research Center, Washington, DC, 2019).

  • Baldwin, J. R. et al. Population vs individual prediction of poor health from results of adverse childhood experiences screening. JAMA Pediatr. 175, 385–393 (2021).

    PubMed 

    Google Scholar
     

  • Marek, S. et al. Towards reproducible brain-wide association studies. Preprint at bioRxiv https://doi.org/10.1101/2020.08.21.257758 (2020).

  • Mansueto, A. C., Wiers, R., van Weert, J. C., Schouten, B. C. & Epskamp, S. Investigating the feasibility of idiographic network models (2020).

  • Kelley, S. & Gillan, C. Using language in social media posts to study the network dynamics of depression longitudinally. Nat. Commun. 13, 1–11 (2022).


    Google Scholar
     

  • Fiesler, C. & Proferes, N. “Participant” perceptions of Twitter research ethics. Soc. Media+ Soc. 4, 2056305118763366 (2018).


    Google Scholar
     

  • Chancellor, S., Birnbaum, M., Caine, E., Silenzio, V. & De Choudhury, M. A taxonomy of ethical tensions in inferring mental health states from social media. In Proc. Conference on Fairness, Accountability, and Transparency (FAT*) (ACM, 2019).

  • Ford, E., Curlewis, K., Wongkoblap, A. & Curcin, V. Public opinions on using social media content to identify users with depression and target mental health care advertising: mixed methods survey. JMIR Ment. Health 6, e12942 (2019).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Mellon, J. & Prosser, C. Twitter and Facebook are not representative of the general population: Political attitudes and demographics of British social media users. Res. Politics 4, 2053168017720008 (2017).


    Google Scholar
     

  • Yang, J., Morris, M. R., Teevan, J., Adamic, L. A. & Ackerman, M. S. Culture matters: A survey study of social Q&A behavior. In Fifth International AAAI Conference on Weblogs and Social Media. 5, 409–416 (2011).

  • Resnik, P. et al. Beyond LDA: exploring supervised topic modeling for depression-related language in Twitter. In Proc. 2nd Workshop on Computational Linguistics and Clinical Psychology. 99–107 (Association for Computational Linguistics, 2015).

  • Orabi, A. H., Buddhitha, P., Orabi, M. H. & Inkpen, D. Deep learning for depression detection of twitter users. In Proc. Fifth Workshop on Computational Linguistics and Clinical Psychology. 88–97 (Association for Computational Linguistics, 2018).

  • Sun, J., Schwartz, H. A., Son, Y., Kern, M. L. & Vazire, S. The language of well-being: Tracking fluctuations in emotion experience through everyday speech. J. Personal. Soc. Psychol. 118, 364 (2020).


    Google Scholar
     

  • Agrawal, A., Fu, W. & Menzies, T. What is wrong with topic modeling? And how to fix it using search-based software engineering. Inf. Softw. Technol. 98, 74–88 (2018).


    Google Scholar
     

  • Greene, D., O’Callaghan, D. & Cunningham, P. How Many Topics? Stability Analysis for Topic Models. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 498–513 (2014).

  • Jaidka, K., Guntuku, S. & Ungar, L. Facebook versus Twitter: Differences in Self-Disclosure and Trait Prediction. In Twelfth International AAAI Conference on Web and Social Media. (AAAI, 2018).

  • Zung, W. W. A self-rating depression scale. Arch. Gen. Psychiatry 12, 63–70 (1965).

    CAS 
    PubMed 

    Google Scholar
     

  • Mason, O., Linney, Y. & Claridge, G. Short scales for measuring schizotypy. Schizophrenia Res. 78, 293–296 (2005).


    Google Scholar
     

  • Foa, E. B. et al. The Obsessive-Compulsive Inventory: development and validation of a short version. Psychological Assess. 14, 485 (2002).


    Google Scholar
     

  • Garner, D., Olmsted, M., Bohr, Y. & Garfinkel, P. The eating attitudes test: psychometric features. Psychological Med. 12, 871–878 (1982).

    CAS 

    Google Scholar
     

  • Patton, J. H., Stanford, M. S. & Barratt, E. S. Factor structure of the Barratt impulsiveness scale. J. Clin. Psychol. 51, 768–774 (1995).

    CAS 
    PubMed 

    Google Scholar
     

  • Saunders, J. B., Aasland, O. G., Babor, T. F., De La Fuente, J. R. & Grant, M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption‐II. Addiction 88, 791–804 (1993).

    CAS 
    PubMed 

    Google Scholar
     

  • Marin, R. S., Biedrzycki, R. C. & Firinciogullari, S. Reliability and validity of the Apathy Evaluation Scale. Psychiatry Res. 38, 143–162 (1991).

    CAS 
    PubMed 

    Google Scholar
     

  • Liebowitz, M. R. Social phobia. Modern problems of pharmacopsychiatry (1987).

  • Spielberger, C. D. State-trait anxiety inventory for adults (1983).

  • Pennebaker, J. W., Boyd, R. L., Jordan, K. & Blackburn, K. The development and psychometric properties of LIWC2015 (2015).

  • Zou, H. & Hastie, T. Regularization and variable selection via the elastic net. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 67, 301–320 (2005).


    Google Scholar
     

  • Jollans, L. et al. Quantifying performance of machine learning methods for neuroimaging data. NeuroImage 199, 351–365 (2019).

    PubMed 

    Google Scholar
     

  • Dinga, R., Schmaal, L., Penninx, B. W., Veltman, D. J. & Marquand, A. F. Controlling for effects of confounding variables on machine learning predictions. Preprint at bioRxiv https://doi.org/10.1101/2020.08.17.255034 (2020).

  • Rouault, M., Seow, T., Gillan, C. M. & Fleming, S. M. Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance. Biol. Psychiatry 84, 443–451 (2018).

    PubMed 
    PubMed Central 

    Google Scholar
     

  • Seow, T. X. & Gillan, C. M. Transdiagnostic phenotyping reveals a host of metacognitive deficits implicated in compulsivity. Sci. Rep. 10, 1–11 (2020).


    Google Scholar
     

  • Murtagh, F. & Legendre, P. Ward’s hierarchical agglomerative clustering method: which algorithms implement Ward’s criterion? J. Classification 31, 274–295 (2014).


    Google Scholar
     

  • Schwartz, H. A. et al. Towards assessing changes in degree of depression through facebook. In Proc. Workshop on Computational Linguistics and Clinical Psychology. 118–125 (Association for Computational Linguistics, 2014).

  • https://www.nature.com/articles/s41746-022-00576-y

    Next Post

    MI Counselor Tips for Managing Return to Work Stress / Public News Service

    More and more workplaces across Michigan are bringing employees back into the office in-person, and health experts have tips for how to manage any stress and anxiety the transition may bring. Kris Henderson, a counselor at Counseling Solutions of West Michigan, said major life changes can often be stress triggers, […]