[Paper Review] Fuzzy Logic Interpretation of Artificial Neural Networks.

Fenglei Fan, Ge Wang|arXiv (Cornell University)|Jul 4, 2018

Neural Networks and Applications4 citations

TL;DR

This paper proposes a fuzzy logic interpretation of artificial neural networks by leveraging second-order neurons that replace traditional inner products with quadratic functions, enabling direct implementation of fuzzy logic operations like XOR. The key contribution is a systematic categorization of implementable fuzzy operations and statistical analysis showing that deep quadratic networks can be interpreted as deep fuzzy logic systems, enhancing interpretability of deep learning models.

ABSTRACT

Over past several years, deep learning has achieved huge successes in various applications. However, such a data-driven approach is often criticized for lack of interpretability. Recently, we proposed artificial quadratic neural networks consisting of second-order neurons in potentially many layers. In each second-order neuron, a quadratic function is used in the place of the inner product in a traditional neuron, and then undergoes a nonlinear activation. With a single second-order neuron, any fuzzy logic operation, such as XOR, can be implemented. In this sense, any deep network constructed with quadratic neurons can be interpreted as a deep fuzzy logic system. Since traditional neural networks and second-order counterparts can represent each other and fuzzy logic operations are naturally implemented in second-order neural networks, it is plausible to explain how a deep neural network works with a second-order network as the system model. In this paper, we generalize and categorize fuzzy logic operations implementable with individual second-order neurons, and then perform statistical/information theoretic analyses of exemplary quadratic neural networks.

Motivation & Objective

• To address the lack of interpretability in deep learning models by providing a fuzzy logic-based interpretation of artificial neural networks.
• To demonstrate that second-order neurons can naturally implement core fuzzy logic operations such as XOR, which traditional neurons cannot.
• To generalize and categorize the set of fuzzy logic operations realizable using individual second-order neurons.
• To perform statistical and information-theoretic analyses on exemplary quadratic neural networks to validate their interpretability and functional equivalence to fuzzy logic systems.

Proposed method

• Replacing the linear inner product in traditional neurons with a quadratic function to form second-order neurons, enabling non-linear decision boundaries.
• Applying a nonlinear activation function after the quadratic computation to form a fully differentiable second-order neuron unit.
• Using a single second-order neuron to implement fundamental fuzzy logic operations, such as XOR, by tuning the quadratic coefficients.
• Constructing deep networks from these second-order neurons to model complex fuzzy logic systems.
• Performing statistical and information-theoretic analyses on trained quadratic networks to evaluate their behavior and interpretability.
• Establishing a formal correspondence between deep quadratic networks and deep fuzzy logic systems through mathematical and empirical validation.

Experimental results

Research questions

• RQ1 Which fuzzy logic operations can be implemented using a single second-order neuron, and how can they be systematically categorized?
• RQ2 How do second-order neurons enable the representation of non-linear, non-monotonic logical operations such as XOR, which are not representable by standard linear neurons?
• RQ3 To what extent can a deep neural network composed of second-order neurons be interpreted as a deep fuzzy logic system?
• RQ4 What statistical and information-theoretic properties emerge in quadratic neural networks that support their interpretation as fuzzy logic systems?
• RQ5 Can the interpretability of deep learning models be enhanced by modeling them through a second-order neural network framework grounded in fuzzy logic?

Key findings

• A single second-order neuron can implement any fuzzy logic operation, including non-linear and non-monotonic ones such as XOR, by appropriately configuring its quadratic function.
• Second-order neurons provide a natural bridge between artificial neural networks and fuzzy logic systems, enabling a formal interpretation of deep networks as fuzzy logic systems.
• The proposed quadratic network architecture allows for mutual representability between traditional deep neural networks and second-order networks, preserving functional equivalence.
• Statistical and information-theoretic analyses of exemplary quadratic networks confirm their capacity to model complex logical relationships consistent with fuzzy logic principles.
• The framework provides a principled way to interpret deep learning models through fuzzy logic, enhancing transparency without sacrificing representational power.