Bunpei IRIE and Mitsuo KAWATO
Extraction of the Nonlinear Global
Coordinate System of a Manifold by
a Five Layered Hour-Glass Network
Abstract:One of the advantages of the Multi Layered Perceptron (MLP), combined with
Back Propagation (BP) algorithm, is its capability of learning from examples.
On the other hand, Memory Based Reasoning (MBR) is also known by its
learnability from examples, in which method the system memorizes the entire
set of the examples of known input-output correspondence and interpolates
them in order to calculate outputs for unknown inputs. Naturally, there arises
a question whether MLP is a mere variety of MBR where example data are
compressed to some extent. In this paper, we will show that MLP has an
additional property, i.e. the capability of acquiring internal representation from
examples.
To show this, a five layered perceptron is made to learn the identity
mapping from the input layer to the output layer. Input vectors are distributed
on a manifold whose dimension is identical to the number of units in the
compressed representation of the third layer. In this configuration, we show
that the network succeeds in acquiring the global nonlinear coordinate system
which is evidently most suitable for the distribution of the example data. The
way to make use of the result for some applications is also discussed.