Description
Modern deep neural networks comprise many operational layers, such as dense or convolutional layers, which are often collected into blocks.In this work, we introduce a new, wavelet-transform-based network architecture that we call the multi-resolution perceptron: by adding a pooling layer, we create a new network block, the WavPool.
The first step of the multi-resolution perceptron is transforming the data into its multi-resolution decomposition form by convolving the input data with filters of fixed coefficients but increasing size. Following image processing techniques, we are able to make scale and spatial information simultaneously accessible to the network without increasing the size of the data vector. This makes it promising for problems such as particle classification, where both size and spatial relationship of input images is important, and increasing the size of the networks through adding stacked convolutional layers is computationally prohibitive.
WavPool outperforms a similar multilayer perceptron while using fewer parameters, and outperforms a comparable convolutional neural network by 10% on relative accuracy on CIFAR-10.