Reiner Lenz
Unsupervised Learning of Receptive Field
Families on Regular Grids
Abstract:We investigate families of receptive fields (i.e. low-level filter systems) that
receive their inputs from sensors located on a finite, regular grid. We will first
introduce a class of models that describe the behaviour of such systems. We
introduce the representation theory of the dihedral groups to derive some
important properties of such systems that originate in the structure of the
grid (and not in the particular nature of the system). We will show that
representation theory leads directly to algorithms with a structure similar to
those of the FFT. We demonstrate possible applications of the theory in the
field of low-level vision by showing how to construct and analyse different
type of filter families. We will show that the same, universal, coordinate
transformation simplifies all these different approaches.
In the second part of the paper we will discuss learning rules that lead to
systems that learn these filter systems from examples. We will introduce three
different types of systems: The Karhunen-Loeve, the quadratic and the fourth order
learning system. All these systems have the same structure that allows
them to learn in parallel. The KL-system stablizes in states that are linear
combinations of the eigenvectors of the input process. We will then introduce
a new variation of the learning rule, based on second order terms that can
differentiate between eigenvectors that belong to different eigenvalues. Finally
we will introduce an energy function that contains fourth order terms. The
resulting systems can no longer be analysed in terms of the covariance function
of the input process but we will demonstrate empirically that they have a
number of advantages over the ordinary KL-transform based systems. We
will also show that systems that use the group theoretically defined coordinate
transformation as pre-processing procedure perform better than the systems
that work on the original pixel data.