[Paper Review] CSI-Net: Unified Human Body Characterization and Pose Recognition

Fei Wang, Jinsong Han+3 more|arXiv (Cornell University)|Oct 7, 2018

Gait Recognition and Analysis19 citations

TL;DR

CSI-Net is a unified deep learning framework that leverages WiFi Channel State Information (CSI) to jointly perform human body characterization and pose recognition. By transforming raw CSI into spatially-encoded tensors and using a scalable CNN backbone, it achieves state-of-the-art performance in biometrics estimation, person recognition, hand sign recognition, and falling detection with test accuracies up to 98.13% and 97.73% respectively.

ABSTRACT

We build CSI-Net, a unified Deep Neural Network~(DNN), to learn the representation of WiFi signals. Using CSI-Net, we jointly solved two body characterization problems: biometrics estimation (including body fat, muscle, water, and bone rates) and person recognition. We also demonstrated the application of CSI-Net on two distinctive pose recognition tasks: the hand sign recognition (fine-scaled action of the hand) and falling detection (coarse-scaled motion of the body).

Motivation & Objective

• To develop a unified deep learning framework that jointly solves multiple human sensing tasks using WiFi CSI.
• To overcome the limitations of hand-crafted features in CSI-based human sensing by learning end-to-end representations.
• To enable accurate biometrics estimation (body fat, muscle, water, bone) using commodity WiFi devices.
• To achieve high-accuracy recognition of fine-grained hand signs and coarse-grained body motions like falling.
• To establish a benchmark dataset and analysis for CSI-based human sensing with physical insights into signal-body interactions.

Proposed method

• CSI-Net processes complex-valued CSI sequences from 802.11n/g systems into spatially-encoded feature maps using transposed convolutional layers or bilinear interpolation.
• The network employs a scalable CNN architecture with ResNet-18 as the backbone, enabling efficient feature learning across diverse sensing tasks.
• The generation stage transforms raw CSI tensors of size $30 \times 1 \times 1$ into $224 \times 224$ image-like representations for standard CNN processing.
• Bilinear interpolation is used as a lightweight alternative to transposed convolutions, improving efficiency while maintaining high accuracy.
• The feature learning stage extracts hierarchical representations from the spatially-encoded CSI tensors using residual blocks.
• The task stage applies fully connected layers with softmax for classification in person recognition, sign recognition, and falling detection, and regression heads for biometrics estimation.

Experimental results

Research questions

• RQ1 Can a unified deep learning framework effectively learn representations from raw WiFi CSI for diverse human sensing tasks?
• RQ2 How does human body presence and composition affect CSI variance, and can this be leveraged for biometrics estimation?
• RQ3 Can CSI-based deep learning achieve high accuracy in fine-grained action recognition (e.g., hand signs) and coarse-grained motion detection (e.g., falling)?
• RQ4 What is the optimal network architecture and preprocessing strategy (e.g., transposed convolutions vs. interpolation) for CSI-based sensing?
• RQ5 How do different CNN backbones (e.g., ResNet, VGG, Inception) compare in performance and efficiency for CSI-based human sensing?

Key findings

• CSI-Net achieved 93.00% test accuracy in person recognition using ResNet-18, outperforming larger backbones like ResNet-152 and VGG-19.
• The model reached 100% test accuracy in hand sign recognition, demonstrating strong capability in fine-grained action recognition.
• For falling detection, CSI-Net achieved 96.67% test accuracy, with only 1.27% misclassification of falling as standing and 5.41% vice versa.
• Replacing transposed convolutions with bilinear interpolation improved test accuracy to 94.42% in person recognition, showing that simpler preprocessing can be more effective.
• ResNet-18 provided the best balance between model complexity and performance, with only 61MB model size, outperforming larger networks like VGG-19 in accuracy and efficiency.
• The study demonstrates the first successful application of commodity WiFi devices for biometrics estimation, including body fat, muscle, water, and bone rates, with high accuracy.