Using Shallow Semantic Analysis to Implement Automated Quality Assessment of Web Health Care Information

Evidence-based clinical practice guidelines have been widely used as an objective rating instrument for assessing the content quality of health care information on the web. In many previous studies, human raters check the concordance between text content and evidence-based practice guidelines in order to evaluate information accuracy and completeness. However, human rating cannot be a practical solution, particularly when there is an extremely large volume of health care information on the web. This study explores a semantics-based approach to identify health care information content in web documents with reference to evidence-based health care guidelines. With this approach terms and phrases in English are extracted and transformed into semantic concepts and units. Thus, web text is transformed, sentence by sentence, into a semantic representation which computer programs can classify depending on whether the content of a sentence is in concordance with evidence-based guidelines or not. Through aggregating the classification result of all sentences in a web document, computer programs are able to generate for each document a quality score indicating the number of unique evidence-based guidelines that are referred to in the document. In a test using a set of depression treatment web pages and evidence-based clinical guidelines, the quality rating performance of the computer system is shown to be close to human quality rating performance.