TY - JOUR
T1 - Characterizing and predicting the multifaceted nature of quality in educational web resources
AU - Wetzler, Philipp
AU - Bethard, Steven
AU - Leary, Heather
AU - Butcher, Kirsten
AU - Bahreini, Soheil Danesh
AU - Zhao, Jin
AU - Martin, James H.
AU - Sumner, Tamara
PY - 2013/10
Y1 - 2013/10
N2 - Efficient learning from Web resources can depend on accurately assessing the quality of each resource. We present a methodology for developing computational models of quality that can assist users in assessing Web resources. The methodology consists of four steps: 1) a meta-analysis of previous studies to decompose quality into high-level dimensions and low-level indicators, 2) an expert study to identify the key low-level indicators of quality in the target domain, 3) human annotation to provide a collection of example resources where the presence or absence of quality indicators has been tagged, and 4) training of a machine learning model to predict quality indicators based on content and link features of Web resources. We find that quality is a multifaceted construct, with different aspects that may be important to different users at different times. We show that machine learning models can predict this multifaceted nature of quality, both in the context of aiding curators as they evaluate resources submitted to digital libraries, and in the context of aiding teachers as they develop online educational resources. Finally, we demonstrate how computational models of quality can be provided as a service, and embedded into applications such as Web search.
AB - Efficient learning from Web resources can depend on accurately assessing the quality of each resource. We present a methodology for developing computational models of quality that can assist users in assessing Web resources. The methodology consists of four steps: 1) a meta-analysis of previous studies to decompose quality into high-level dimensions and low-level indicators, 2) an expert study to identify the key low-level indicators of quality in the target domain, 3) human annotation to provide a collection of example resources where the presence or absence of quality indicators has been tagged, and 4) training of a machine learning model to predict quality indicators based on content and link features of Web resources. We find that quality is a multifaceted construct, with different aspects that may be important to different users at different times. We show that machine learning models can predict this multifaceted nature of quality, both in the context of aiding curators as they evaluate resources submitted to digital libraries, and in the context of aiding teachers as they develop online educational resources. Finally, we demonstrate how computational models of quality can be provided as a service, and embedded into applications such as Web search.
KW - Algorithms
UR - http://www.scopus.com/inward/record.url?scp=84983584983&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84983584983&partnerID=8YFLogxK
U2 - 10.1145/2533670.2533673
DO - 10.1145/2533670.2533673
M3 - Article
AN - SCOPUS:84983584983
SN - 2160-6455
VL - 3
JO - ACM Transactions on Interactive Intelligent Systems
JF - ACM Transactions on Interactive Intelligent Systems
IS - 3
M1 - A15
ER -