Estimating Local Function Complexity via Mixture of Gaussian Processes.

Published on 2019in arXiv: Learning
Danny Panknin2
Estimated H-index: 2
(Technical University of Berlin),
Shinichi Nakajima13
Estimated H-index: 13
+ 1 AuthorsKlaus-Robert Müller82
Estimated H-index: 82
Real world data often exhibit inhomogeneity, e.g., skewed distribution, non-uniform complexity of the target function and uneven noise level over the input space. In this paper, we cope with inhomogeneity by explicitly estimating the locally optimal kernel bandwidth as a function. Specifically, we propose Spatially Adaptive Bandwidth Estimation in Regression (SABER), which employs the mixture of experts consisting of multinomial kernel logistic regression as a gate and Gaussian process regression models as experts. SABER can estimate the optimal kernel bandwidth much more accurately and stably than existing kernel width estimation methods, and shows comparable prediction performance with deep neural networks in quantum chemistry applications. Drawing parallels to the theory of locally linear smoothing, we derive an estimate to the local function complexity. Local function complexity can be used for model interpretation, active learning and Bayesian optimization. Those aspects are also demonstrated in quantum chemistry experiments and fluid dynamics simulations.
  • References (42)
  • Citations (0)
#1Sebastian Lapuschkin (Heinrich Hertz Institute)H-Index: 9
#2Stephan Wäldchen (Technical University of Berlin)H-Index: 1
Last.Klaus-Robert Müller (Technical University of Berlin)H-Index: 82
view all 6 authors...
#1Stephan Lenz (Braunschweig University of Technology)
#2Manfred Krafczyk (Braunschweig University of Technology)H-Index: 33
Last.Zhaoli Guo (HUST: Huazhong University of Science and Technology)
view all 5 authors...
#1Maria Peifer (UPenn: University of Pennsylvania)
#2F O Chamon Luiz (UPenn: University of Pennsylvania)
Last.Alejandro Ribeiro (UPenn: University of Pennsylvania)H-Index: 33
view all 4 authors...
#1Kristof T. Sch "utt (Technical University of Berlin)H-Index: 10
#2Huziel E. Sauceda (MPG: Max Planck Society)H-Index: 7
Last.Klaus-Robert Müller (Technical University of Berlin)H-Index: 82
view all 5 authors...
Aug 10, 2015 in KDD (Knowledge Discovery and Data Mining)
#1Richard A. Reviewer-Caruana (Microsoft)H-Index: 40
#2Yinjun Lou (LinkedIn)H-Index: 2
Last.Noémie Elhadad (Columbia University)H-Index: 30
view all 6 authors...
#1Raghunathan Ramakrishnan (University of Basel)H-Index: 11
#2Pavlo O. Dral (MPG: Max Planck Society)H-Index: 11
Last.O. Anatole von Lilienfeld (Argonne National Laboratory)H-Index: 28
view all 4 authors...
#1Ziyi Chen (PKU: Peking University)H-Index: 2
#2Jinwen Ma (PKU: Peking University)H-Index: 3
Last.Yatong Zhou (PKU: Peking University)H-Index: 1
view all 3 authors...
Cited By0