[Paper Review] GPU-based Commonsense Paradigms Reasoning for Real-Time Query Answering and Multimodal Analysis

Ha Nguyen Tran, Erik Cambria|arXiv (Cornell University)|Jul 14, 2018

Sentiment Analysis and Opinion Mining3 citations

TL;DR

This paper proposes GPU-accelerated frameworks for real-time commonsense reasoning and multimodal sentiment analysis, leveraging subgraph similarity search and rule-based SPARQL querying on knowledge graphs. It achieves orders-of-magnitude speedup over CPU-based systems using optimized GPU kernels and data structures like Bloom filters, with 78% accuracy on YouTube sentiment analysis and linear scalability on large-scale datasets.

ABSTRACT

We utilize commonsense knowledge bases to address the problem of real- time multimodal analysis. In particular, we focus on the problem of multimodal sentiment analysis, which consists in the simultaneous analysis of different modali- ties, e.g., speech and video, for emotion and polarity detection. Our approach takes advantages of the massively parallel processing power of modern GPUs to enhance the performance of feature extraction from different modalities. In addition, in order to ex- tract important textual features from multimodal sources we generate domain-specific graphs based on commonsense knowledge and apply GPU-based graph traversal for fast feature detection. Then, powerful ELM classifiers are applied to build the senti- ment analysis model based on the extracted features. We conduct our experiments on the YouTube dataset and achieve an accuracy of 78% which outperforms all previous systems. In term of processing speed, our method shows improvements of several orders of magnitude for feature extraction compared to CPU-based counterparts.

Motivation & Objective

• To address the performance and scalability bottlenecks in real-time commonsense reasoning and query answering on large-scale knowledge bases.
• To exploit the massive parallelism of GPUs to accelerate subgraph similarity search and rule-based inference in commonsense knowledge graphs.
• To enhance multimodal sentiment analysis by integrating GPU-accelerated feature extraction and commonsense-based text feature detection.
• To develop a co-processing model balancing GPU and CPU workloads for optimal performance in reasoning pipelines.
• To extend the system to support richer query types, including wildcards and regular expressions, and higher expressiveness in description logics.

Proposed method

• Proposes GpSense, a GPU-based framework using a filtering-and-joining strategy for subgraph similarity search, with multi-level graph compression to reduce memory footprint.
• Employs GPU-optimized primitives such as sort, merge, prefix scan, and compaction to execute reasoning trees in a bottom-up fashion on GPUs.
• Introduces a GPU-based Bloom filter and sorting algorithm to efficiently handle triple duplication during backward-chaining inference.
• Develops gSparql, a GPU-accelerated SPARQL query processor that maps RDF triples into GPU-executable kernels using stream processing primitives.
• Applies GPU-ELM (Extreme Learning Machine) for fast multimodal sentiment classification using features extracted from text, audio, and video in parallel.
• Constructs domain-specific commonsense graphs from SenticNet to extract salient textual features, which are then fused at feature and decision levels for improved accuracy.

Experimental results

Research questions

• RQ1 Can GPU acceleration significantly improve the performance of subgraph similarity search in commonsense knowledge graphs?
• RQ2 How can rule-based SPARQL query processing be effectively mapped onto GPU architectures to achieve low-latency inference?
• RQ3 To what extent can GPU-accelerated feature extraction and commonsense-enhanced text representation improve multimodal sentiment analysis accuracy and speed?
• RQ4 What are the key optimization techniques that enable linear scalability and high throughput in GPU-based commonsense reasoning systems?
• RQ5 How can GPU-CPU co-processing and advanced data layouts be designed to support complex queries and reduce communication overhead?

Key findings

• GpSense achieves linear scalability and outperforms state-of-the-art CPU-based methods by several orders of magnitude in subgraph similarity search.
• The system demonstrates a 78% accuracy on the YouTube multimodal sentiment analysis dataset, surpassing previous systems.
• GPU-based feature extraction for audio, video, and text components achieves speedups of multiple orders of magnitude compared to CPU-based implementations.
• The use of GPU-optimized Bloom filters and sorting reduces triple duplication overhead and improves inference efficiency in rule-based reasoning.
• Multi-level graph compression reduces memory usage while preserving query accuracy, enabling efficient processing on large-scale knowledge bases.
• The framework scales effectively on both Gowalla and Enron datasets, confirming robustness across diverse real-world knowledge graphs.