TY - GEN
T1 - Measuring the latency of depression detection in social media
AU - Sadeque, Farig
AU - Xu, Dongfang
AU - Bethard, Steven
N1 - Funding Information:
This work was supported by National Institutes of Health grant R01GM114355 from the National Institute of General Medical Sciences (NIGMS). The computations were done in systems supported by the National Science Foundation under Grant No. 1228509. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or National Science Foundation.
Publisher Copyright:
© 2018 Association for Computing Machinery.
PY - 2018/2/2
Y1 - 2018/2/2
N2 - Detecting depression is a key public health challenge, as almost 12% of all disabilities can be attributed to depression. Computational models for depression detection must prove not only that can they detect depression, but that they can do it early enough for an intervention to be plausible. However, current evaluations of depression detection are poor at measuring model latency. We identify several issues with the currently popular ERDE metric, and propose a latency-weighted F1 metric that addresses these concerns. We then apply this evaluation to several models from the recent eRisk 2017 shared task on depression detection, and show how our proposed measure can better capture system differences.
AB - Detecting depression is a key public health challenge, as almost 12% of all disabilities can be attributed to depression. Computational models for depression detection must prove not only that can they detect depression, but that they can do it early enough for an intervention to be plausible. However, current evaluations of depression detection are poor at measuring model latency. We identify several issues with the currently popular ERDE metric, and propose a latency-weighted F1 metric that addresses these concerns. We then apply this evaluation to several models from the recent eRisk 2017 shared task on depression detection, and show how our proposed measure can better capture system differences.
KW - Depression
KW - Latency
KW - Neural networks
KW - Social media
UR - http://www.scopus.com/inward/record.url?scp=85046900602&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85046900602&partnerID=8YFLogxK
U2 - 10.1145/3159652.3159725
DO - 10.1145/3159652.3159725
M3 - Conference contribution
AN - SCOPUS:85046900602
T3 - WSDM 2018 - Proceedings of the 11th ACM International Conference on Web Search and Data Mining
SP - 495
EP - 503
BT - WSDM 2018 - Proceedings of the 11th ACM International Conference on Web Search and Data Mining
PB - Association for Computing Machinery, Inc
T2 - 11th ACM International Conference on Web Search and Data Mining, WSDM 2018
Y2 - 5 February 2018 through 9 February 2018
ER -