[Paper Review] Deep Generalized Convolutional Sum-Product Networks

Jos van de Wolfshaar, Andrzej Pronobis|arXiv (Cornell University)|Feb 16, 2019

Visual Attention and Saliency Detection22 references4 citations

TL;DR

This paper introduces Deep Generalized Convolutional Sum-Product Networks (DGC-SPNs), a novel SPN architecture that generalizes convolutional operations in product layers using a new parameterization of strides and dilations, enabling overlapping patches while preserving probabilistic validity. DGC-SPNs achieve state-of-the-art performance in image inpainting and classification across multiple datasets, outperforming existing SPN models and offering a scalable, GPU-optimized implementation via a Keras/TensorFlow library.

ABSTRACT

Sum-Product Networks (SPNs) are hierarchical, graphical models that combine benefits of deep learning and probabilistic modeling. SPNs offer unique advantages to applications demanding exact probabilistic inference over high-dimensional, noisy inputs. Yet, compared to convolutional neural nets, they struggle with capturing complex spatial relationships in image data. To alleviate this issue, we introduce Deep Generalized Convolutional Sum-Product Networks (DGC-SPNs), which encode spatial features in a way similar to CNNs, while preserving the validity of the probabilistic SPN model. As opposed to existing SPN-based image representations, DGC-SPNs allow for overlapping convolution patches through a novel parameterization of dilations and strides, resulting in significantly improved feature coverage and feature resolution. DGC-SPNs substantially outperform other SPN architectures across several visual datasets and for both generative and discriminative tasks, including image inpainting and classification. These contributions are reinforced by the first simple, scalable, and GPU-optimized implementation of SPNs, integrated with the widely used Keras/TensorFlow framework. The resulting model is fully probabilistic and versatile, yet efficient and straightforward to apply in practical applications in place of traditional deep nets.

Motivation & Objective

• To address the limitation of existing SPN architectures in capturing complex spatial relationships in image data, similar to CNNs.
• To enable overlapping convolution patches in SPN product layers without violating validity constraints (completeness and decomposability).
• To improve feature resolution and coverage in SPN-based image representations by generalizing the parameterization of strides and dilation rates.
• To develop a scalable, efficient, and GPU-optimized implementation of SPNs integrated with Keras/TensorFlow for practical deployment.
• To demonstrate the superiority of DGC-SPNs across both generative and discriminative visual tasks, including image inpainting and classification.

Proposed method

• Introduce a novel parameterization of convolutional operations in SPN product layers that allows overlapping patches by generalizing strides and dilation rates.
• Maintain SPN validity through completeness and decomposability by ensuring consistent scopes across sum and product nodes.
• Use weighted sum and product operations to hierarchically combine local features into global representations, preserving exact probabilistic inference.
• Integrate unweighted sum inputs (USI) with hard EM to reduce bias in path selection and improve performance on missing data tasks.
• Implement the DGC-SPN architecture in a new library, libspn-keras, built on Keras and TensorFlow for scalability and ease of use.
• Train the model using standard deep learning optimization techniques (e.g., Adam, dropout) while retaining full probabilistic semantics.

Experimental results

Research questions

• RQ1 Can overlapping convolution patches in SPN product layers improve feature resolution and coverage without violating probabilistic validity?
• RQ2 Does a generalized parameterization of strides and dilations in SPNs lead to better performance on image recognition and reconstruction tasks?
• RQ3 Can DGC-SPNs achieve state-of-the-art results in both generative and discriminative image tasks compared to existing SPN architectures?
• RQ4 How does the use of unweighted sum inputs (USI) with hard EM affect performance and path selection bias in SPNs?
• RQ5 To what extent does the new GPU-optimized, Keras/TensorFlow-based implementation of SPNs enhance scalability and practical usability?

Key findings

• DGC-SPNs achieved the lowest MSE on image inpainting tasks across all datasets, with a 12.7% improvement on Olivetti (735 vs. 804) and a 12.8% improvement on Caltech 101 (2801 vs. 3216) when using USI.
• On MNIST, DGC-SPNs achieved 98.66% accuracy in image classification, outperforming all prior SPN-based methods, including RAT-SPN (98.19%) and Prometheus (98.37%).
• On Fashion MNIST, DGC-SPNs achieved 90.74% accuracy, surpassing the next best method (RAT-SPN at 89.52%).
• The use of unweighted sum inputs (USI) with hard EM significantly reduced path selection bias, improving performance across all datasets and tasks.
• DGC-SPNs demonstrated superior scalability by using a single shared SPN stack across all classes, unlike prior methods requiring sub-SPNs per class, which scale poorly with increasing class counts.
• The libspn-keras library enables efficient, scalable, and GPU-accelerated training and inference, facilitating reproducibility and broader adoption of SPN-based models.