Estimating Local Function Complexity via Mixture of Gaussian Processes.

Published on Feb 27, 2019in arXiv: Learning
Danny Panknin2
Estimated H-index: 2
(Technical University of Berlin),
Shinichi Nakajima14
Estimated H-index: 14
+ 1 AuthorsKlaus-Robert Müller92
Estimated H-index: 92
Real world data often exhibit inhomogeneity, e.g., the noise level, the sampling distribution or the complexity of the target function may change over the input space. In this paper, we try to isolate local function complexity in a practical, robust way. This is achieved by first estimating the locally optimal kernel bandwidth as a functional relationship. Specifically, we propose Spatially Adaptive Bandwidth Estimation in Regression (SABER), which employs the mixture of experts consisting of multinomial kernel logistic regression as a gate and Gaussian process regression models as experts. Using the locally optimal kernel bandwidths, we deduce an estimate to the local function complexity by drawing parallels to the theory of locally linear smoothing. We demonstrate the usefulness of local function complexity for model interpretation and active learning in quantum chemistry experiments and fluid dynamics simulations.
  • References (46)
  • Citations (0)
📖 Papers frequently viewed together
4 Authors (Ye Kuang, ..., Renxin Zhong)
2017NeurIPS: Neural Information Processing Systems
11 Citations
2017CDC: Conference on Decision and Control
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1Sebastian Lapuschkin (Heinrich Hertz Institute)H-Index: 10
#2Stephan Wäldchen (Technical University of Berlin)H-Index: 1
Last. Klaus-Robert Müller (Technical University of Berlin)H-Index: 92
view all 6 authors...
Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly intelligent behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be obli...
37 CitationsSource
#1Stephan Lenz (Braunschweig University of Technology)H-Index: 1
#2Manfred Krafczyk (Braunschweig University of Technology)H-Index: 36
Last. Zhaoli Guo (HUST: Huazhong University of Science and Technology)H-Index: 1
view all 5 authors...
Abstract Gas-kinetic schemes (GKS) have been developed as a kinetic Finite-Volume approach to computational fluid dynamics. The GKS a priori allows to obtain approximate solutions of the fully compressible Navier-Stokes equations. In our contribution we show simulation results of compressible natural convection at large temperature differences and low Mach numbers beyond the applicable range of the Boussinesq approximation. The simulations were performed on non-uniform quadrilateral and unstruct...
2 CitationsSource
#1Maria Peifer (UPenn: University of Pennsylvania)H-Index: 1
#2F O Chamon Luiz (UPenn: University of Pennsylvania)H-Index: 1
Last. Alejandro Ribeiro (UPenn: University of Pennsylvania)H-Index: 39
view all 4 authors...
Reproducing Kernel Hilbert Space (RKHS)-based methods are widely used in signal processing and machine learning applications. Yet, they suffer from a parameter selection issue: selecting the RKHS in which to operate (or even the kernel parameter) is often a significant challenge. Moreover, since the RKHS determines properties such as shape and smoothness of the learned function, its choice affects the effectiveness of these techniques. Likewise, due to the homogeneous smoothness of functions pro...
1 CitationsSource
#1Stefan Chmiela (Technical University of Berlin)H-Index: 6
#2Huziel E. SaucedaH-Index: 8
Last. Alexandre Tkatchenko (University of Luxembourg)H-Index: 50
view all 4 authors...
Molecular dynamics (MD) simulations employing classical force fields constitute the cornerstone of contemporary atomistic modeling in chemistry, biology, and materials science. However, the predictive power of these simulations is only as good as the underlying interatomic potential. Classical potentials often fail to faithfully capture key quantum effects in molecules and materials. Here we enable the direct construction of flexible molecular force fields from high-level ab initio calculations ...
33 CitationsSource
#1Kristof T. Sch "utt (Technical University of Berlin)H-Index: 12
#2Huziel E. Sauceda (MPG: Max Planck Society)H-Index: 8
Last. Klaus R. Muller (Technical University of Berlin)H-Index: 12
view all 5 authors...
Deep learning has led to a paradigm shift in artificial intelligence, including web, text, and image search, speech recognition, as well as bioinformatics, with growing impact in chemical physics. Machine learning, in general, and deep learning, in particular, are ideally suitable for representing quantum-mechanical interactions, enabling us to model nonlinear potential-energy surfaces or enhancing the exploration of chemical compound space. Here we present the deep learning architecture SchNet ...
91 CitationsSource
#1Wiktor Pronobis (Technical University of Berlin)H-Index: 3
#2Alexandre Tkatchenko (University of Luxembourg)H-Index: 50
Last. Klaus-Robert Müller (Technical University of Berlin)H-Index: 92
view all 3 authors...
Machine learning (ML) based prediction of molecular properties across chemical compound space is an important and alternative approach to efficiently estimate the solutions of highly complex many-electron problems in chemistry and physics. Statistical methods represent molecules as descriptors that should encode molecular symmetries and interactions between atoms. Many such descriptors have been proposed; all of them have advantages and limitations. Here, we propose a set of general two-body and...
15 CitationsSource
#1Stefan ChmielaH-Index: 6
Last. Klaus-Robert Müller (Technical University of Berlin)H-Index: 92
view all 6 authors...
Using conservation of energy—a fundamental property of closed classical and quantum mechanical systems—we develop an efficient gradient-domain machine learning (GDML) approach to construct accurate molecular force fields using a restricted number of samples from ab initio molecular dynamics (AIMD) trajectories. The GDML implementation is able to reproduce global potential energy surfaces of intermediate-sized molecules with an accuracy of 0.3 kcal mol −1 for energies and 1 kcal mol −1 A −1 for a...
111 CitationsSource
Aug 13, 2016 in KDD (Knowledge Discovery and Data Mining)
#1Marco Túlio de Freitas Ribeiro (UW: University of Washington)H-Index: 14
#2Sameer Singh (UW: University of Washington)H-Index: 21
Last. Carlos Guestrin (UW: University of Washington)H-Index: 65
view all 3 authors...
Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model. Such understanding also provides insights into the model, which can be used to transform an untrustworthy model or prediction into a trustworthy one. In this work, we propose LIME, a novel explanation techn...
881 CitationsSource
Aug 10, 2015 in KDD (Knowledge Discovery and Data Mining)
#1Richard A. Reviewer-Caruana (Microsoft)H-Index: 45
#2Yin Lou (LinkedIn)H-Index: 10
Last. Noémie Elhadad (Columbia University)H-Index: 32
view all 6 authors...
In machine learning often a tradeoff must be made between accuracy and intelligibility. More accurate models such as boosted trees, random forests, and neural nets usually are not intelligible, but more intelligible models such as logistic regression, naive-Bayes, and single decision trees often have significantly worse accuracy. This tradeoff sometimes limits the accuracy of models that can be applied in mission-critical applications such as healthcare where being able to understand, validate, ...
293 CitationsSource
#1Raghunathan Ramakrishnan (University of Basel)H-Index: 11
#2Pavlo O. Dral (MPG: Max Planck Society)H-Index: 11
Last. O. Anatole von Lilienfeld (Argonne National Laboratory)H-Index: 31
view all 4 authors...
Computational de novo design of new drugs and materials requires rigorous and unbiased exploration of chemical compound space. However, large uncharted territories persist due to its size scaling combinatorially with molecular size. We report computed geometric, energetic, electronic, and thermodynamic properties for 134k stable small organic molecules made up of CHONF. These molecules correspond to the subset of all 133,885 species with up to nine heavy atoms (CONF) out of the GDB-17 chemical u...
167 CitationsSource
Cited By0