Representation Learning for Deviation Detection - Revision history

Slawek at 20:20, 9 September 2019

2019-09-09T20:20:11Z

Slawek at 18:00, 14 October 2018

2018-10-14T18:00:22Z

Slawek at 12:00, 3 January 2017

2017-01-03T12:00:47Z

Slawek at 11:59, 3 January 2017

2017-01-03T11:59:57Z

Slawek at 11:59, 3 January 2017

2017-01-03T11:59:18Z

Slawek at 11:58, 3 January 2017

2017-01-03T11:58:05Z

Slawek at 07:54, 8 December 2016

2016-12-08T07:54:58Z

Yuafan at 22:22, 25 October 2016

2016-10-25T22:22:21Z

Yuafan at 22:22, 25 October 2016

2016-10-25T22:22:03Z

Yuafan at 22:21, 25 October 2016

2016-10-25T22:21:32Z

@@ Line 2: / Line 2: @@
 |Summary=Optimise Echo State Networks for time series forecasting and reconstruction. Propose methods, e.g. objective functions, to train Echo State Networks for deviation detection
 |Keywords=Representation learning, Deviation detection, Echo State Network, Optimization, Differential evolution
-|TimeFrame=Fall 2018
+|TimeFrame=Fall 2019
 |References=Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.

@@ Line 2: / Line 2: @@
 |Summary=Optimise Echo State Networks for time series forecasting and reconstruction. Propose methods, e.g. objective functions, to train Echo State Networks for deviation detection
 |Keywords=Representation learning, Deviation detection, Echo State Network, Optimization, Differential evolution
-|TimeFrame=Winter 2016 / Spring 2017
+|TimeFrame=Fall 2018
 |References=Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.
@@ Line 26: / Line 26: @@
 |Prerequisites=Artificial Intelligence and Learning Systems courses; good knowledge of machine learning and neural networks; programming skills for implementing machine learning algorithms
 |Supervisor=Sławomir Nowaczyk, Yuantao Fan
 |Level=Master
 |Status=Open

@@ Line 4: / Line 4: @@
 |TimeFrame=Winter 2016 / Spring 2017
 |References=Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.
 |Prerequisites=Artificial Intelligence and Learning Systems courses; good knowledge of machine learning and neural networks; programming skills for implementing machine learning algorithms
 |Supervisor=Sławomir Nowaczyk, Yuantao Fan

@@ Line 4: / Line 4: @@
 |TimeFrame=Winter 2016 / Spring 2017
 |References=Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.
 |Prerequisites=Artificial Intelligence and Learning Systems courses; good knowledge of machine learning and neural networks; programming skills for implementing machine learning algorithms
 |Supervisor=Sławomir Nowaczyk, Yuantao Fan

@@ Line 3: / Line 3: @@
 |Keywords=Representation learning, Deviation detection, Echo State Network, Optimization, Differential evolution
 |TimeFrame=Winter 2016 / Spring 2017
-|References=[1] Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.
+|References=Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.
 [2] Jaeger, Herbert. "Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the" echo state network" approach. GMD-Forschungszentrum Informationstechnik, 2002.

@@ Line 5: / Line 5: @@
 |References=[1] Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.
-[2] Jaeger, Herbert. Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the" echo state network" approach. GMD-Forschungszentrum Informationstechnik, 2002.
+[2] Jaeger, Herbert. "Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the" echo state network" approach. GMD-Forschungszentrum Informationstechnik, 2002.
 [3] Jaeger, Herbert, et al. "Optimization and applications of echo state networks with leaky-integrator neurons." Neural networks 20.3 (2007): 335-352.
@@ Line 12: / Line 12: @@
 [5] Wang, Lin, Zhigang Wang, and Shan Liu. "An effective multivariate time series classification approach using echo state network and adaptive differential evolution algorithm." Expert Systems with Applications 43 (2016): 237-249.
 |Prerequisites=Artificial Intelligence and Learning Systems courses; good knowledge of machine learning and neural networks; programming skills for implementing machine learning algorithms
 |Supervisor=Sławomir Nowaczyk, Yuantao Fan
@@ Line 30: / Line 42: @@
 . Perform deviation detection (e.g. COSMO method) on both synthetic data and data from real world application based on Echo State Networks and other data representations. Compare and analyze the performance.

@@ Line 18: / Line 18: @@
 }}
 Echo State Network (ESN) is a special class of recurrent neural networks that can be used for supervised learning, e.g. time series forecasting, pattern recognition and deviation detection. The main idea of ESN is to drive a large random, fixed neural network that induce input signal into each neurons. The nonlinear responses of all neurons are combined by a trainable linear combination to a desired output. The network captures temporal features of time series data. Compared to other recurrent neural networks, it can be trained much faster. ESN are characterized by a number of parameters (e.g. network architecture, spectral radius, type of neurons and input scaling etc.). In order to achieve desired performance, these parameters need to be optimized and, preferably, in an efficient way.
 There are many applications [0,1,2] ESN can be applied to. ESN can be used for general learning tasks, e.g. time series forecasting, or as representation of the data. In [3,4] ESN is employed to classify time series for fault detection. Similarly, in [6], air pressure signals are encoded into ESNs and anomalies are detected using an unsupervised deviation detection method, called Consensus Self-Organizing Models (COSMO) method. One key feature of COSMO method is the ability to capture and encode characteristics of the signals by using different representations. Generic representation that can be adapted to different type of signals and captures various characteristics is preferred for anomaly detection. Paper [6] demonstrated that ESNs are promising for this purpose. How to optimize the parameters of ESNs for deviation detection can be further investigated.

@@ Line 25: / Line 25: @@
 . Use ESNs as self-organizing data representation (can be trained without external supervision) for unsupervised deviation detection. Determine what are the criteria for ESNs to learn interesting characteristics of the time series for deviation detection. Note that, for example, minimizing the reconstruction error does not necessary to be the objective function when train ESNs, but rather how ESNs can be trained to generalize various properties that characterize the observed system.
 . Perform deviation detection (e.g. COSMO method) on both synthetic data and data from real world application based on Echo State Networks and other data representations. Compare and analyze the performance.
 References: