The proposed network architecture and loss design.Abstract
Surface parameter estimation is an essential field in computer games and movies. An exact representation of a real-world surface allows for a higher degree of realism. Capturing or artistically creating these materials is a time-consuming process. We propose a method which utilizes an encoder-decoder Convolutional Neural Network (CNN) to extract parameters for the Bidirectional Reflectance Distribution Function (BRDF) automatically from a sparse sample set. This is done by implementing a differentiable renderer, which allows for a loss backpropagation of rendered images. This photometric loss is essential because defining a numerical BRDF distance metric is difficult. A second loss is added, which compares the parameters maps directly. Therefore, the statistical properties of the BRDF model are learned, which reduces artifacts in the predicted parameters. This dual loss principal improves the result of the network significantly. Opposed to previous means this method retrieves information of the whole surface as spatially varying BRDF (SVBRDF) parameters with a sufficiently high resolution for intended real-world usage. The capture process for materials only requires five known light positions with a fixed camera position. This reduces the scanning time drastically, and a material sample can be obtained in seconds with an automated system.
Mark Boss
Research Scientist
I’m a researcher at Stability AI with research interests in the intersection of machine learning and computer graphics.