1. Shape recovery demo - synthetic objects and images -
Start this demo by right-clicking on "Demo 1" : Hit the "return" key or use the mouse to rotate the original and the recovered shapes. Right click to
see the next example when you are satisfied that you have seen how well the model recovered the 3D shape from the specific 2D image used. Press ESC
to exit this demo. The first six examples in this demo make use of the same "original" shape. Its recovery was made from different 2D images of this
"original" shape." The 3D shape recovered was almost the same for all of the 2D images that were used to recover it. Note also that the entire 3D shape
is recovered, namely, both the visible surfaces in front and the invisible surfaces in back were recovered despite the fact that the model was given 2D
images from which hidden edges were removed. This demo includes three different 3D shapes, with six 2D images for each. The last 3D shape shown in this
demo represents a chair. This 3D chair was represented by 3D points and recovered from the 2D images of these points. In all of the examples shown in
this demo, the model was given the contours or points, as well as the information about which features are symmetric in 3D space and which contours are
planar in 3D space. In other words, figure-ground organization was provided to the model to make it possible for the 3D shape to be recovered from its
2D images.
This demo showed that once figure-ground organization is provided to the model, it can recover the shape of a 3D object, represented by a line drawing,
from a
variety of the 2D images that would be produced if the object was viewed from almost any viewing direction. Now, consider whether this model can be applied to
real images of real objects? It can because most real images of real objects can be represented quite well by line drawings. In other words, this model will be
able to recover the 3D structure of real objects as well as it can recover their representations in line drawings.
Note that even though 3D properties of individual points and features, such as depth and surface orientation, are always ambiguous in a single 2D image,
shape
is almost never ambiguous because shape, unlike other perceptual properties, such as color, is complex. This fact explains why it is easier to recover 3D shape
than to recover the depths of points and the orientations of surfaces.
This claim, which would have been considered paradoxical in 1709 (Berkeley) or even in 1912 (Wertheimer), does not seem paradoxical today
because a computational
model can recover 3D shape from 2D shape, something that cannot be done if one tries to do this by working with points. We like to think that the success
of our
computational model is a good example of what Gestalt Psychologists had in mind when they said that "the whole is different from the sum of its parts."
2. Shape recovery demo - real images segmentated
by hand -
see Demo 1 to find out how to start and run this demo. In this demo, the contours in the 2D image, given to the model, were extracted by an unskilled
human hand. The model was also given information about which features were symmetric in 3D space and which contours were planar in 3D space. Press "c"
to toggle the contours and "i" to toggle the 2D image. Press "pause" to stop the rotation, and "s" to synchronize the rotation, after you changed the
3D orientation of one of the 3D shapes by using mouse. The "chair" was recovered from six different 2D images.
Note that the 3D shape could be recovered very well from a 2D image even when the 2D information about the contours in the image, which was provided to the model,
was very crude. This means that our model's recovery of 3D shape is quite robust in the presence of the noise and errors in the 2D image. The human visual system
does a much better job "extracting contours" than the unskilled human, who drew the contours used for 3D recovery in this demo.
The important message illustrated by this demo is that the spatially global aspects of the 2D image (its 2D shape) is the important determinant of 3D shape
recovery. Spatial details, such as exact positions of points and magnitudes of curvatures of contours, are irrelevant. We can now claim that "the whole is
not only different from its parts, it is also more important than its parts."
3. Shape recovery demo - real images segmentated
automatically* -
instructions are the same as for Demo 2. Again, the model was given information about which features were symmetric in 3D space and which contours
were planar in 3D space. In this demo, our symmetry constraint was applied to more contours than in Demo 2.
The contours, extracted automatically, and used for the recoveries shown is this demo, were obviously more accurate than those extracted by hand (Demo 2),
and as one might expect, the 3D shapes recovered are more accurate, too. The recovered 3D shapes are more accurate primarily because the symmetry constraint
was applied to more edges than in Demo 2.
Symmetry (mirror, rotational and translational) is probably the most important shape constraint ("prior") because it restricts the family of possible 3D
interpretations dramatically. A 3D interpretation of a 2D image of N unrelated points is characterized by N degrees of freedom. The free parameters are the
depths of the points. But, when the points form a mirror-symmetric configuration in 3D space, and the skewed (distorted) symmetry is detected in the 2D image,
the 3D interpretation is characterized by only one degree of freedom (see the description of the symmetry constraint in our computational model, linked above).
Note that the symmetry constraint is used in our model to make up for the information that is missing from the 2D image, not to compress the 2D image, as others
have done. Our use of symmetry is better because the primary task of the visual system is to see 3D shapes not to code 2D images.
When all is said and done, it seems likely that our main contribution consists of pointing out that most, if not all objects "out there" are characterized by at
least one type of symmetry. This is almost surely the case with respect to many of the objects that are important to us. Symmetry of objects "out there" is almost
never perfect. This is the case either because the objects are not exactly symmetrical or because parts of symmetrical objects can move independently (for example,
human and animal bodies). But, the human observer can easily detect partial and/or approximate symmetry. Even a little bit of symmetry goes a very long way. Without
3D symmetry there is no 3D shape, no percept of 3D shape and no shape constancy. The fundamental importance of symmetry cannot be overstated.
* (Additional examples of 3D shape recovery with variety of natural 3D shapes, such as cars, planes, boats, bicycles, insects, and birds can be
viewed on Yunfeng's and on Tada's web sites. These
examples were shown at our posters, as well as at the Demo Night, at the VSS 2008 Meeting in Naples, FL).
Acknowledgment: The demos on this site were prepared by Yunfeng Li.
For more demos visit
ViPER Lab web page >>