Publications

Machine learning
Numerical Analysis

Scaling Gaussian Process Regression with Derivatives
D Eriksson, K Dong, EH Lee, H Nickisch, D Bindel, AG Wilson
Advances in Neural Information Processing Systems 32 (NIPS), 2018
Download     Abstract

Abstract:

Gaussian processes (GPs) with derivatives are useful in many applications, including Bayesian optimization, implicit surface reconstruction, and terrain reconstruction. Fitting a GP to function values and derivatives at $n$ points in $d$ dimensions requires linear solves and log determinants with an ${n(d+1) \times n(d+1)}$ positive definite matrix -- leading to prohibitive $\mathcal{O}(n^3d^3)$ computations for standard direct methods. We propose iterative solvers using fast $\mathcal{O}(nd)$ matrix-vector multiplications (MVMs), together with pivoted Cholesky preconditioning that cuts the iterations to convergence by several orders of magnitude, allowing for fast kernel learning and prediction. Our approaches, together with dimensionality reduction, enables Bayesian optimization with derivatives to scale to high-dimensional problems and large evaluation budgets.

Machine learning
Numerical Analysis

Scalable Log Determinants for Gaussian Process Kernel Learning
K Dong, D Eriksson, H Nickisch, D Bindel, AG Wilson
Advances in Neural Information Processing Systems 31 (NIPS), 2017
Download     Abstract

Abstract:

For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an $n \times n$ positive definite matrix, and its derivatives – leading to prohibitive $\mathcal{O}(n^3)$ computations. We propose novel $\mathcal{O}(n)$ approaches to estimating these quantities from only fast matrix vector mul- tiplications (MVMs). These stochastic approximations are based on Chebyshev, Lanczos, and surrogate models, and converge quickly even for kernel matrices that have challenging spectra. We leverage these approximations to develop a scalable Gaussian process approach to kernel learning. We find that Lanczos is generally superior to Chebyshev for kernel learning, and that a surrogate approach can be highly efficient and accurate with popular kernels.

Comp. Geometry
Numerical Analysis
HPC

Fast exact shortest distance queries for massive point clouds
D Eriksson, E Shellshear
Graphical Models 84, 28-37
Download     Abstract

Abstract:

This paper describes a new efficient algorithm for the rapid computation of exact shortest distances between a point cloud and another object (e.g. triangulated, point-based, etc.) in three dimensions. It extends the work presented in Eriksson and Shellshear (2014) where only approximate distances were computed on a simplification of a massive point cloud. Here, the fast computation of the exact shortest distance is achieved by pruning large subsets of the point cloud known not to be closest to the other object. The approach works for massive point clouds even with a small amount of RAM and is able to provide real time performance. Given a standard PC with only 8GB of RAM, this resulted in real-time shortest distance computations of 15 frames per second for a point cloud having 1 billion points in three dimensions.

Numerical PDEs
Numerical Analysis
HPC
Geodesy

Tropospheric delay ray tracing applied in VLBI analysis
D Eriksson, DS MacMillan, JM Gipson
Journal of Geophysical Research: Solid Earth 119 (12), 9156-9170
Download     Abstract

Abstract:

Tropospheric delay modeling error continues to be one of the largest sources of error in VLBI (very long baseline interferometry) analysis. For standard operational solutions, we use the VMF1 elevation-dependent mapping functions derived from European Centre for Medium-Range Weather Forecasts data. These mapping functions assume that tropospheric delay at a site is azimuthally symmetric. As this assumption is not true, we have instead determined the ray trace delay along the signal path through the troposphere for each VLBI quasar observation. We determined the troposphere refractivity fields from the pressure, temperature, specific humidity, and geopotential height fields of the NASA Goddard Space Flight Center Goddard Earth Observing System version 5 numerical weather model. When applied in VLBI analysis, baseline length repeatabilities were improved compared with using the VMF1 mapping function model for 72% of the baselines and site vertical repeatabilities were better for 11 of 13 sites during the 2 week CONT11 observing period in September 2011. When applied to a larger data set (2011–2013), we see a similar improvement in baseline length and also in site position repeatabilities for about two thirds of the stations in each of the site topocentric components.

Numerical PDEs
Numerical Analysis
Geodesy

Continental hydrology loading observed by VLBI measurements
D Eriksson, DS MacMillan
Journal of Geodesy 88 (7), 675-690
Download     Abstract

Abstract:

Variations in continental water storage lead to loading deformation of the crust with typical peak-to-peak variations at very long baseline interferometry (VLBI) sites of 3–15 mm in the vertical component and 1–2 mm in the horizontal component. The hydrology signal at VLBI sites has annual and semi-annual components and clear interannual variations. We have calculated the hydrology loading series using mass loading distributions derived from the global land data assimilation system (GLDAS) hydrology model and alternatively from a global grid of equal-area gravity recovery and climate experiment (GRACE) mascons. In the analysis of the two weekly VLBI 24-h R1 and R4 network sessions from 2003 to 2010 the baseline length repeatabilities are reduced in 79 % (80 %) of baselines when GLDAS (GRACE) loading corrections are applied. Site vertical coordinate repeatabilities are reduced in about 80 % of the sites when either GLDAS or GRACE loading is used. In the horizontal components, reduction occurs in 70–80 % of the sites. Estimates of the annual site vertical amplitudes were reduced for 16 out of 18 sites if either loading series was applied. We estimated loading admittance factors for each site and found that the average admittances were 1.01 ± 0.05 for GRACE and 1.39 ± 0.07 for GLDAS. The standard deviations of the GRACE admittances and GLDAS admittances were 0.31 and 0.68, respectively. For sites that have been observed in a set of sufficiently temporally dense daily sessions, the average correlation between VLBI vertical monthly averaged series and GLDAS or GRACE loading series was 0.47 and 0.43, respectively.

Comp. Geometry
Numerical Analysis
HPC

Approximate distance queries for path-planning in massive point clouds
D Eriksson, E Shellshear
Informatics in Control, Automation and Robotics (ICINCO), 2014
Download     Abstract

Abstract:

In this paper, algorithms have been developed that are capable of efficiently pre-processing massive point clouds for the rapid computation of the shortest distance between a point cloud and other objects (e.g. triangulated, point-based, etc.). This is achieved by exploiting fast distance computations between specially structured subsets of a simplified point cloud and the other object. This approach works for massive point clouds even with a small amount of RAM and was able to speed up the computations, on average, by almost two orders of magnitude. Given only 8 GB of RAM, this resulted in shortest distance computations of 30 frames per second for a point cloud originally having 1 billion points. The findings and implementations will have a direct impact for the many companies that want to perform path-planning applications through massive point clouds since the algorithms are able to produce real-time distance computations on a standard PC.