Gaussian Process Framework for Deep Neural Networks
Recent work (Garriga-Alonso et al., 2018) has shown deep convolutional neural networks(DNNs) can be approximated by a shallow Gaus-sian processes (GP) with much fewer parameters.A lot of features in modern convolutional neuralnetwork (CNN) have not been considered in thiswork. In this paper, to extend the flexibility ofthe transformation from modern neural networkarchitecture to shallow Gaussian process, a frame-work for DNNs is introduced. An average poolingoperation and a concatenation operation are de-rived to support densely-connection structures tofit in Garriga-Alonso et al.’s architecture by trans-forming them into simple matrix multiplication.The newly derived DenseNet-GP significantly re-duce the time for calculating the kernel matrixfor the GP while having comparable accuracy inclassifying images in CIFAR10.
This is a course project for ORIE 6741 Bayesian Machine Learning