Scalable score computation for learning multinomial Bayesian networks over distributed data

Praveen Rao, Anas Katib, Kobus Barnard, Charles Kamhoua, Kevin Kwiat, Laurent Njilla

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

In this paper, we focus on the problem of learning a Bayesian network over distributed data stored in a commodity cluster. Specifically, we address the challenge of computing the scoring function over distributed data in a scalable manner, which is a fundamental task during learning. We propose a novel approach designed to achieve: (a) scalable score computation using the principle of gossiping; (b) lower resource consumption via a probabilistic approach for maintaining scores using the properties of a Markov chain; and (c) effective distribution of tasks during score computation (on large datasets) by syncrgistically combining well-known hashing techniques. Through theoretical analysis, we show that our approach is superior to a MapReduce-style computation in terms of communication bandwidth. Further, it is superior to the batch- style processing of MapReduce for recomputing scores when new data are available.

Original languageEnglish (US)
Title of host publicationWS-17-01
Subtitle of host publicationArtificial Intelligence and Operations Research for Social Good; WS-17-02: Artificial Intelligence, Ethics, and Society; WS-17-03: Artificial Intelligence for Connected and Automated Vehicles; WS-17-04: Artificial Intelligence for Cyber Security; WS-17-05: Artificial Intelligence for Smart Grids and Buildings; WS-17-06: Computer Poker and Imperfect Information Games; WS-17-07: Crowdsourcing, Deep Learning and Artificial Intelligence Agents; WS-17-08: Distributed Machine Learning; WS-17-09: Joint Workshop on Health Intelligence; WS-17-10: Human-Aware Artificial Intelligence; WS-17-11: Human-Machine Collaborative Learning; WS-17-12: Knowledge-Based Techniques for Problem Solving and Reasoning; WS-17-13: Plan, Activity, and Intent Recognition; WS-17-14: Symbolic Inference and Optimization; WS-17-15: What's Next for AI in Games?
PublisherAI Access Foundation
Pages498-504
Number of pages7
ISBN (Electronic)9781577357865
StatePublished - 2017
Event31st AAAI Conference on Artificial Intelligence, AAAI 2017 - San Francisco, United States
Duration: Feb 4 2017Feb 5 2017

Publication series

NameAAAI Workshop - Technical Report
VolumeWS-17-01 - WS-17-15

Other

Other31st AAAI Conference on Artificial Intelligence, AAAI 2017
Country/TerritoryUnited States
CitySan Francisco
Period2/4/172/5/17

ASJC Scopus subject areas

  • General Engineering

Fingerprint

Dive into the research topics of 'Scalable score computation for learning multinomial Bayesian networks over distributed data'. Together they form a unique fingerprint.

Cite this