Photometric Stereo by LearningJean-Denis Durou∗, Yvain Queau∗, Vincent Charvillat∗

∗IRIT-VORTEX, ENSEEIHT, Toulouse, France

Abstract— In this work, we solve photometric stereo by learning.The basic idea consists in inserting a sphere in the scene to bereconstructed, and in covering both the sphere and the scene withthe same painting. The images of the sphere being consideredas learning data, the knowledge of the scene reflectance is notrequired any more to implement photometric stereo. In addition,the self-shadows become meaningful. We show how a PrincipalComponents Analysis (PCA) allows us to reach the same resultsas Hertzmann et al. [1] but faster. On the other hand, we showhow to detect the cast-shadows of the scene in the case where nosuch shadow occurs on the sphere. We also show how to extendour approach to non-painted scenes.

I. INTRODUCTION

Amongst the different 3D-reconstruction techniques, shape-from-shading is often discarded, according to its limited per-formances. An important improvement is reached using severalimages of the scene taken from the same viewpoint, but underdifferent illuminations. This technique, known as photometricstereo, was proposed first by Woodham [1]. In the last years, ithas become rather popular, due to very accurate results obtainedat small scales [2].

II. ACCELERATION OF HERTZMANN’S METHOD

Our starting point is a recent paper by Hertzmann et al.[3], in which photometric stereo is solved by learning. Moreprecisely, a sphere is inserted in the scene to be reconstructed(cf. Fig. 1). If n illuminations are used, each pixel P of thescene is given the same normal as the point on the spherewhich is the most similar to P , according to their n greylevelsread in the n images. Our first contribution is to make a PCA,which allows us to reduce the problem dimension from n top < n, thus accelerating Hertzmann’s method with comparableaccuracy (cf. Fig. 2).

Fig. 1. Real images of a bottle (1176×398 pixels) and of a sphere (332×328pixels) covered with the same painting, seen under n = 8 different lights [3].

Fig. 2. 3D-reconstructions of the bottle using photometric stereo on the imagesof Fig. 1: on the left, Hertzmann’s method (CPU = 608 s); on the right, ourmethod which resorts to the 3 first principal components (CPU = 238 s).

III. IMPROVEMENTS OF HERTZMANN’S METHOD

We also show how to generalize this method to the cast-shadowed pixels, which are usually considered as outliers.Finally, we show how to avoid the need to cover both the sceneand the sphere with the same painting (see Figs. 3 and 4).

Fig. 3. Synthetic images of a teapot and of a sphere seen under n = 5different lights. The sphere has a uniform albedo, while the teapot is textured.

Fig. 4. Results obtained applying our method to the images of Fig. 3: on theleft, estimated albedo; on the right, reconstructed surface.

REFERENCES

[1] R. J. Woodham, “Photometric Method for Determining Surface Orienta-tion from Multiple Images,” Opt. Engin., 19(1):139–144, 1980.

[2] C. Hernandez, G. Vogiatzis, and R. Cipolla, “Overcoming shadows in3-source photometric stereo,” IEEE Trans. Patt. Anal. Mach. Intell.,33(2):419–426, February 2011.

[3] A. Hertzmann and S. M. Seitz, “Example-Based Photometric Stereo:Shape Reconstruction with General, Varying BRDFs,” IEEE Trans. Patt.Anal. Mach. Intell., 27(8):1254–1264, August 2005.