ATR Human Information Processing Research Labs
Understanding the Mechanism for Space Perception
Hirohiko Kaneko
Humans can perceive the three-dimensional structure of the outside world from
the two-dimensional images that are projected on the retinas of the two eyes.
In this process, the very slight differences between the images formed in the
right and left eyes (binocular disparity) is an important source of in formation.
This article introduces some of our recent studies investigating the mechanisms
that process binocular disparity for human three-dimensional perception.
1. Introduction
Because our eyes are about 6 cm apart, there are slight differences between the
images that are projected on the retinas of the two eyes. Moreover, these differences
correspond geometrically to the spatial structure of the outside world. That these
differences serve as critical information to create the perception of depth has
long been understood and demonstrated experimentally.1
Many tree-dimensional (3-D) display systems that employ this human depth perception
mechanism have been proposed since the invention of the mirrored stereoscope in
the 19th century. Currently, the technique of using light-polarizing filters and
shutters to present separate images to the left and right eyes is often used.
Head-mounted equipment that uses two small display devices, one for each eye,
have also been made practical and are being used in virtual reality systems.
Hearing about such practical 3-D displays that are based on the geometrical correspondence
between binocular disparity and the spatial structure of the outside world, one
might wonder if anymore research into the mechanism for processing binocular disparity
is really needed. As we explain below, however, the mechanism in our brain does
not simply convert binocular disparity into the geometrically corresponding degrees
of depth to create a perception of the location of objects in space. Rather, the
binocular disparity data is encoded in a more intelligent manner. Unfortunately,
the method by which this encoding is done is not fully understood at this time.
If it were understood, however, it would be possible to solve many of the problems
that arise with current 3-D displays, which aim simply for geometrically correct
imaging. Those problems include fatigue when used for long periods of time, doubling
of the data volume, and an imperfect sense of depth. Overcoming these problems
would be a great advance in 3-D display technology.
2. Binocular Corresponding Points (reference points for
detecting binocular disparity)
In determining the magnitude of binocular disparity, it is important to first
know the points of correspondence on the left and right retinas, which are the
positions at which there is zero binocular disparity. These binocular corresponding
points are easily determined by applying geometry. Considering the two eyeballs
to be exactly the same and to be globes that have respective longitudes and latitudes
in the same way that the globe of the earth has, points that have the same longitude
and latitude are considered to be corresponding points. When the two eyes are
looking at the same point in space, as shown by the dotted lines in Figure
1, all of the points that lie on the circle that passes through the fixation
point and the nodal points of the eyes and on the vertical line that passes through
the fixation point, have zero binocular disparity, i.e., that their images are
formed on corresponding points.
It is known, however, that for humans, the points of correspondence on the left
and right retinas deviate in the horizontal directions from the geometrical points
of correspondence. The corresponding points for humans are the points at which
the images are "perceived" as being at the same location.
We attempted to create a map of the corresponding points for the entire retina
by using the same kind of criterion for judgement. The results showed that the
psychophysical corresponding points deviate from the geometrical corresponding
points in the vertical directions as well as in the horizontal directions and
that the horizontal deviation in the periphery have the same trend as in the center.
From that data, we inferred that the surface for which all of the points form
an image on the binocular corresponding points of the retinas has a shape that
is curved forward on the outside and inclined toward the distance at the top (solid
line in figure 1). This surface is the
surface of zero disparity, i.e., the reference surface for detecting binocular
disparity.2
But why does the surface of zero disparity have this shape?
Speculate as we might, this may be determined by experience. The shape and distance
of the surface of zero disparity seems to correspond to the locations of objects
that are of visual interest in our everyday lives (i.e., the location of objects
that we look at as we read or work at a desk). (Note: the test subjects were researchers
who enjoyed studying from a young age and spent a lot of time at their work.)
In other words, because there is a high probability that objects that are looked
at are in the vicinity of this surface, the distribution of cells that detect
binocular disparity around the points that correspond to this surface would allow
effective processing of binocular disparity.
3. Vertical Disparity Distribution and Depth Perception
That the geometrical correspondence between differences in left-eye and right-eye
images in the horizontal direction (horizontal binocular disparity) and the depth
or objects in the outside world is due to the horizontal separation of the two
eyes is easily understood. There is also a geometrical correspondence of differences
between left and right images in the vertical direction (vertical disparity) and
the distance and direction of the visual object. This can be understood intuitively
when we consider that the image from an object that is displaced to the right
with respect to our face, for example, is larger in the right eye than in the
left eye because of the shorter distance to the right eye. Thus, each plane in
front of our eyes has a distribution of horizontal and vertical disparity that
is characteristic to its distance from the plane of the eyes (Figure
2).
We investigated the relationship between this distribution of disparity and depth
perception. The results revealed that when only the horizontal component of the
distribution shown in figure 2 is examined,
we see that a surface that curves toward the distance on the outside is perceived.
Examining only the vertical component of the distribution, we see that a surface
that curves forward toward the viewer on the outside is perceived. When both components
are present together, the plane is seen flat. That is to say, the depth from horizontal
binocular disparity and the depth from vertical binocular disparity mutually cancel
out and the plane can be perceived as a plane, regardless of its distance.
Investigating the relationship between vertical disparity and depth perception
in greater detail, we found that the depth shape of the perceived plane is actually
changed even by impossible vertical binocular disparity distributions. For example,
from a pattern in which objects on the right side of the screen are larger in
the left-eye image than in the right-eye image (Figure
3, upper left), a plane whose outer parts curve into the distance is perceived
(figure 3, lower left), and from a pattern
where the vertical disparity gradually changes from the right side of the screen
to the left side of the screen (Figure 3,
upper right), a plane whose upper part is inclined toward the viewer (Figure
3, lower right) is perceived.3,4
In these cases, there is no simple correspondence of the disparity distribution
and the perceived depth as there is for the horizontal binocular disparity. For
the case shown on the left side of Figure
3, the mean vertical binocular disparity within a given region corresponds
to the "inclination" of that region. Furthermore, for the case shown on the right
side of Figure 3, the difference between
the mean value of the vertical "shear" disparity of the entire visual field and
the individual horizontal disparities contributes to the degree of perceived depth.
This vertical disparity pattern also causes rotational movement of the eyeball
(rotation around the optical axis of the eye).
4. Conclusion
As the examples presented here show, the mechanism for reconstruction of three-dimensional
spatial structure from binocular disparity is not necessarily geometrically logical
or efficient. However, even though a mechanism that creates a sense of depth even
from disparities distributions that are not actually possible might seem, at a
glance, to hold no significance, it probably does offer humans advantages in being
more efficient and robust against measurement errors for the purpose of living
and moving around in three-dimensional space.
With the belief that some day, when the human spatial perception mechanisms is
understood completely, current 3-D displays, which reproduce two two-dimensional
images that are geometrically correct, will be replaced by true 3-D displays that
reproduce perceptually correct three-dimensional structures, we will continue
research in this area.
The research introduced here was conducted in collaboration with Philip Grove
(York Univ.) and Yoshiki Fukunaga (Nara Institute of Science and Technology) in
Department 5 at the ATR Human Information Processing Research Laboratories.
Reference
