Toward AI Research Methodology: Three Case Studies in Evaluation

Paul R. Cohen, Adele E. Howe

    Research output: Contribution to journalArticlepeer-review

    30 Scopus citations


    The roles of evaluation in empirical artificial intelligence (Al) research are described, in an idealized cyclic model and in the context of three case studies. The case studies illustrate pitfalls in evaluation and the contributions of evaluation at all stages of the research cycle. Evaluation methods are contrasted with those of the behavioral sciences, and it is concluded that AI must define and refine its own methods. To this end, several experiment “schemas” and many specific evaluation criteria are described; recommendations are offered in the hope of encouraging the development and practice of evaluation methods in AI.

    Original languageEnglish (US)
    Pages (from-to)634-646
    Number of pages13
    JournalIEEE Transactions on Systems, Man and Cybernetics
    Issue number3
    StatePublished - 1989

    ASJC Scopus subject areas

    • Engineering(all)


    Dive into the research topics of 'Toward AI Research Methodology: Three Case Studies in Evaluation'. Together they form a unique fingerprint.

    Cite this