Flow of Information in Feed-Forward Denoising Neural Networks

Pejman Khadivi; Ravi Tandon; Naren Ramakrishnan

doi:10.1109/ICCI-CC.2018.8482098

Flow of Information in Feed-Forward Denoising Neural Networks

Pejman Khadivi, Ravi Tandon, Naren Ramakrishnan

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Scopus citations

Abstract

Due to inaccuracies in data acquisition, time series data often suffer from noise and instability which leads to inaccurate data mining results. The ability to handle noisy time series data is thus critical in many data-driven real-time applications. Using the locality feature of time series, feed-forward deep neural networks has been effectively used for time series denoising. In this paper, in order to understand the underling behavior of denoising neural networks, we use an information theoretic approach to study the flow of information and to determine how the entropy of information changes between consecutive layers. We develop analytical bounds for multi-layer feed-forward deep neural networks deployed in time series denoising. Numerical experiments support our theoretical conclusions.

Original language	English (US)
Title of host publication	Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018
Editors	Newton Howard, Sam Kwong, Yingxu Wang, Jerome Feldman, Bernard Widrow, Phillip Sheu
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	166-173
Number of pages	8
ISBN (Electronic)	9781538633601
DOIs	https://doi.org/10.1109/ICCI-CC.2018.8482098
State	Published - Oct 4 2018
Event	17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018 - Berkeley, United States Duration: Jul 16 2018 → Jul 18 2018

Publication series

Name	Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018

Conference

Conference	17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018
Country/Territory	United States
City	Berkeley
Period	7/16/18 → 7/18/18

ASJC Scopus subject areas

Artificial Intelligence
Information Systems
Cognitive Neuroscience

Access to Document

10.1109/ICCI-CC.2018.8482098

Cite this

Khadivi, P., Tandon, R., & Ramakrishnan, N. (2018). Flow of Information in Feed-Forward Denoising Neural Networks. In N. Howard, S. Kwong, Y. Wang, J. Feldman, B. Widrow, & P. Sheu (Eds.), Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018 (pp. 166-173). Article 8482098 (Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCI-CC.2018.8482098

Flow of Information in Feed-Forward Denoising Neural Networks. / Khadivi, Pejman; Tandon, Ravi; Ramakrishnan, Naren.
Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018. ed. / Newton Howard; Sam Kwong; Yingxu Wang; Jerome Feldman; Bernard Widrow; Phillip Sheu. Institute of Electrical and Electronics Engineers Inc., 2018. p. 166-173 8482098 (Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Khadivi, P, Tandon, R & Ramakrishnan, N 2018, Flow of Information in Feed-Forward Denoising Neural Networks. in N Howard, S Kwong, Y Wang, J Feldman, B Widrow & P Sheu (eds), Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018., 8482098, Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018, Institute of Electrical and Electronics Engineers Inc., pp. 166-173, 17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018, Berkeley, United States, 7/16/18. https://doi.org/10.1109/ICCI-CC.2018.8482098

Khadivi P, Tandon R, Ramakrishnan N. Flow of Information in Feed-Forward Denoising Neural Networks. In Howard N, Kwong S, Wang Y, Feldman J, Widrow B, Sheu P, editors, Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 166-173. 8482098. (Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018). doi: 10.1109/ICCI-CC.2018.8482098

Khadivi, Pejman ; Tandon, Ravi ; Ramakrishnan, Naren. / Flow of Information in Feed-Forward Denoising Neural Networks. Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018. editor / Newton Howard ; Sam Kwong ; Yingxu Wang ; Jerome Feldman ; Bernard Widrow ; Phillip Sheu. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 166-173 (Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018).

@inproceedings{ac7649efc4444ca3a4fa62957d7fc75f,

title = "Flow of Information in Feed-Forward Denoising Neural Networks",

abstract = "Due to inaccuracies in data acquisition, time series data often suffer from noise and instability which leads to inaccurate data mining results. The ability to handle noisy time series data is thus critical in many data-driven real-time applications. Using the locality feature of time series, feed-forward deep neural networks has been effectively used for time series denoising. In this paper, in order to understand the underling behavior of denoising neural networks, we use an information theoretic approach to study the flow of information and to determine how the entropy of information changes between consecutive layers. We develop analytical bounds for multi-layer feed-forward deep neural networks deployed in time series denoising. Numerical experiments support our theoretical conclusions.",

author = "Pejman Khadivi and Ravi Tandon and Naren Ramakrishnan",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018 ; Conference date: 16-07-2018 Through 18-07-2018",

year = "2018",

month = oct,

day = "4",

doi = "10.1109/ICCI-CC.2018.8482098",

language = "English (US)",

series = "Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "166--173",

editor = "Newton Howard and Sam Kwong and Yingxu Wang and Jerome Feldman and Bernard Widrow and Phillip Sheu",

booktitle = "Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018",

}

TY - GEN

T1 - Flow of Information in Feed-Forward Denoising Neural Networks

AU - Khadivi, Pejman

AU - Tandon, Ravi

AU - Ramakrishnan, Naren

PY - 2018/10/4

Y1 - 2018/10/4

N2 - Due to inaccuracies in data acquisition, time series data often suffer from noise and instability which leads to inaccurate data mining results. The ability to handle noisy time series data is thus critical in many data-driven real-time applications. Using the locality feature of time series, feed-forward deep neural networks has been effectively used for time series denoising. In this paper, in order to understand the underling behavior of denoising neural networks, we use an information theoretic approach to study the flow of information and to determine how the entropy of information changes between consecutive layers. We develop analytical bounds for multi-layer feed-forward deep neural networks deployed in time series denoising. Numerical experiments support our theoretical conclusions.

AB - Due to inaccuracies in data acquisition, time series data often suffer from noise and instability which leads to inaccurate data mining results. The ability to handle noisy time series data is thus critical in many data-driven real-time applications. Using the locality feature of time series, feed-forward deep neural networks has been effectively used for time series denoising. In this paper, in order to understand the underling behavior of denoising neural networks, we use an information theoretic approach to study the flow of information and to determine how the entropy of information changes between consecutive layers. We develop analytical bounds for multi-layer feed-forward deep neural networks deployed in time series denoising. Numerical experiments support our theoretical conclusions.

UR - http://www.scopus.com/inward/record.url?scp=85056459075&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056459075&partnerID=8YFLogxK

U2 - 10.1109/ICCI-CC.2018.8482098

DO - 10.1109/ICCI-CC.2018.8482098

M3 - Conference contribution

AN - SCOPUS:85056459075

T3 - Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018

SP - 166

EP - 173

BT - Proceedings of 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018

A2 - Howard, Newton

A2 - Kwong, Sam

A2 - Wang, Yingxu

A2 - Feldman, Jerome

A2 - Widrow, Bernard

A2 - Sheu, Phillip

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th IEEE International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018

Y2 - 16 July 2018 through 18 July 2018

ER -

Flow of Information in Feed-Forward Denoising Neural Networks

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this