1. Shape recovery demo - synthetic objects and images - Start this demo by right-clicking on "Demo 1" : Hit the "return" key or use the mouse to rotate the original and the recovered shapes. Right click to see the next example when you are satisfied that you have seen how well the model recovered the 3D shape from the specific 2D image used. Press ESC to exit this demo. The first six examples in this demo make use of the same "original" shape. Its recovery was made from different 2D images of this "original" shape." The 3D shape recovered was almost the same for all of the 2D images that were used to recover it. Note also that the entire 3D shape is recovered, namely, both the visible surfaces in front and the invisible surfaces in back were recovered despite the fact that the model was given 2D images from which hidden edges were removed. This demo includes three different 3D shapes, with six 2D images for each. The last 3D shape shown in this demo represents a chair. This 3D chair was represented by 3D points and recovered from the 2D images of these points. In all of the examples shown in this demo, the model was given the contours or points, as well as the information about which features are symmetric in 3D space and which contours are planar in 3D space. In other words, figure-ground organization was provided to the model to make it possible for the 3D shape to be recovered from its 2D images.

This demo showed that once figure-ground organization is provided to the model, it can recover the shape of a 3D object, represented by a line drawing, from a variety of the 2D images that would be produced if the object was viewed from almost any viewing direction. Now, consider whether this model can be applied to real images of real objects? It can because most real images of real objects can be represented quite well by line drawings. In other words, this model will be able to recover the 3D structure of real objects as well as it can recover their representations in line drawings.

Note that even though 3D properties of individual points and features, such as depth and surface orientation, are always ambiguous in a single 2D image, shape is almost never ambiguous because shape, unlike other perceptual properties, such as color, is complex. This fact explains why it is easier to recover 3D shape than to recover the depths of points and the orientations of surfaces.

This claim, which would have been considered paradoxical in 1709 (Berkeley) or even in 1912 (Wertheimer), does not seem paradoxical today because a computational model can recover 3D shape from 2D shape, something that cannot be done if one tries to do this by working with points. We like to think that the success of our computational model is a good example of what Gestalt Psychologists had in mind when they said that "the whole is different from the sum of its parts."

2. Shape recovery demo - real images segmentated by hand - see Demo 1 to find out how to start and run this demo. In this demo, the contours in the 2D image, given to the model, were extracted by an unskilled human hand. The model was also given information about which features were symmetric in 3D space and which contours were planar in 3D space. Press "c" to toggle the contours and "i" to toggle the 2D image. Press "pause" to stop the rotation, and "s" to synchronize the rotation, after you changed the 3D orientation of one of the 3D shapes by using mouse. The "chair" was recovered from six different 2D images.

Note that the 3D shape could be recovered very well from a 2D image even when the 2D information about the contours in the image, which was provided to the model, was very crude. This means that our model's recovery of 3D shape is quite robust in the presence of the noise and errors in the 2D image. The human visual system does a much better job "extracting contours" than the unskilled human, who drew the contours used for 3D recovery in this demo.

The important message illustrated by this demo is that the spatially global aspects of the 2D image (its 2D shape) is the important determinant of 3D shape recovery. Spatial details, such as exact positions of points and magnitudes of curvatures of contours, are irrelevant. We can now claim that "the whole is not only different from its parts, it is also more important than its parts."

3. Shape recovery demo - real images segmentated automatically^* - instructions are the same as for Demo 2. Again, the model was given information about which features were symmetric in 3D space and which contours were planar in 3D space. In this demo, our symmetry constraint was applied to more contours than in Demo 2.

The contours, extracted automatically, and used for the recoveries shown is this demo, were obviously more accurate than those extracted by hand (Demo 2), and as one might expect, the 3D shapes recovered are more accurate, too. The recovered 3D shapes are more accurate primarily because the symmetry constraint was applied to more edges than in Demo 2.

Symmetry (mirror, rotational and translational) is probably the most important shape constraint ("prior") because it restricts the family of possible 3D interpretations dramatically. A 3D interpretation of a 2D image of N unrelated points is characterized by N degrees of freedom. The free parameters are the depths of the points. But, when the points form a mirror-symmetric configuration in 3D space, and the skewed (distorted) symmetry is detected in the 2D image, the 3D interpretation is characterized by only one degree of freedom (see the description of the symmetry constraint in our computational model, linked above).

Note that the symmetry constraint is used in our model to make up for the information that is missing from the 2D image, not to compress the 2D image, as others have done. Our use of symmetry is better because the primary task of the visual system is to see 3D shapes not to code 2D images.

When all is said and done, it seems likely that our main contribution consists of pointing out that most, if not all objects "out there" are characterized by at least one type of symmetry. This is almost surely the case with respect to many of the objects that are important to us. Symmetry of objects "out there" is almost never perfect. This is the case either because the objects are not exactly symmetrical or because parts of symmetrical objects can move independently (for example, human and animal bodies). But, the human observer can easily detect partial and/or approximate symmetry. Even a little bit of symmetry goes a very long way. Without 3D symmetry there is no 3D shape, no percept of 3D shape and no shape constancy. The fundamental importance of symmetry cannot be overstated.

^* (Additional examples of 3D shape recovery with variety of natural 3D shapes, such as cars, planes, boats, bicycles, insects, and birds can be viewed on Yunfeng's and on Tada's web sites. These examples were shown at our posters, as well as at the Demo Night, at the VSS 2008 Meeting in Naples, FL).

Acknowledgment: The demos on this site were prepared by Yunfeng Li.

For more demos visit ViPER Lab web page >>