Learning to Learn and Sample BRDFs

Computer Graphics Forum (Proceedings of Eurographics 2023)


Chen Liu, Michael Fischer, Tobias Ritschel

University College London

PDF Slides Code Suppl. arXiv Publisher

meta-sampling_teaser
Equal-time comparison between a neural BRDF model fitted without meta-learning (top), fitted with meta-learning (second row) and fitted with our meta-sampling (third row).

Abstract

We propose a method to accelerate the joint process of physically acquiring and learning neural BRDF models. While BRDF learning alone can be accelerated by meta-learning, acquisition remains slow as it relies on a mechanical process. We show that meta-learning can be extended to optimize the physical sampling pattern, too. After our method has been meta-trained for a set of fully-sampled BRDFs, it is able to quickly train on new BRDFs with up to five orders of magnitude fewer physical acquisition samples at similar quality. Our approach also extends to other linear and non-linear BRDF models, which we show in an extensive evaluation.

Algorithm

In the right is our proposed method, which meta-trains the locations of samples until they become the most informative.

The orange snippet meta-trains the model parameters, and the blue snippet meta-trains the sample locations.

  • Learn: Run SGD fitting with the given model parameters and samples on one task, in a differentiable way.

  • Evaluate: Evaluate the given model parameters over one task, in a differentiable way.

algo
A short version of the algorithm.
Check the paper for the full explanation.

Exemplary Results

Learned patterns

diffuse
2D projections (\(\theta_d-\theta_h\)) of our meta-sampling patterns at n = 8 for all models.
The NJR5 pattern is also included for reference.

These samples above are deemed as optimal for BRDF acquisition by our method.

We provide all optimal samples (from n=1 to n=1024) trained by ourselves, for off-shelf usages and to people who don’t want to train again. You can download them at the code repo.

Same # of samples

diffuse
Reconstructions by 8 samples for blue-rubber (diffuse)
glossy
Reconstructions by 32 samples for silver-paint (glossy)

Increasing # of samples

glossy
results of using our proposed meta-sampling for the training of each model (vertical) on the BRDF two-layer-gold for an increasing number of samples (horizontal).

Acknowledgement

We appreciate valuable comments from Vlastimil Havran, Stavros Diolatzis, and Gilles Rainer. We also thank Meta Reality Labs for funding a part of this project.

Citation

@article{liuLearningLearnSample2023,
  title={Learning to Learn and Sample BRDFs},
  author={Liu, Chen and Fischer, Michael and Ritschel, Tobias},
  journal={Computer Graphics Forum (Proceedings of Eurographics)},
  year={2023},
  volume={42},
  number={2},
  pages={201--211},
  doi={10.1111/cgf.14754},
}