[Paper Review] Data-Driven Stochastic Models and Policies for Energy Harvesting Sensor Communications

Meng‐Lin Ku, Yan Chen+1 more|arXiv (Cornell University)|May 28, 2014

Energy Harvesting in Wireless NetworksEngineering28 references17 citations

TL;DR

This paper proposes a data-driven Markov decision process (MDP) framework for optimizing transmission policies in solar-powered wireless sensor networks, using real solar irradiance data to model energy harvesting dynamics and adapt transmission parameters in real time. The approach achieves significant net bit rate gains—up to 1.5× higher than baseline policies—by leveraging belief-based solar state estimation and threshold-structured on-off transmission policies that exploit channel diversity.

ABSTRACT

Energy harvesting from the surroundings is a promising solution to perpetually power-up wireless sensor communications. This paper presents a data-driven approach of finding optimal transmission policies for a solar-powered sensor node that attempts to maximize net bit rates by adapting its transmission parameters, power levels and modulation types, to the changes of channel fading and battery recharge. We formulate this problem as a discounted Markov decision process (MDP) framework, whereby the energy harvesting process is stochastically quantized into several representative solar states with distinct energy arrivals and is totally driven by historical data records at a sensor node. With the observed solar irradiance at each time epoch, a mixed strategy is developed to compute the belief information of the underlying solar states for the choice of transmission parameters. In addition, a theoretical analysis is conducted for a simple on-off policy, in which a predetermined transmission parameter is utilized whenever a sensor node is active. We prove that such an optimal policy has a threshold structure with respect to battery states and evaluate the performance of an energy harvesting node by analyzing the expected net bit rate. The design framework is exemplified with real solar data records, and the results are useful in characterizing the interplay that occurs between energy harvesting and expenditure under various system configurations. Computer simulations show that the proposed policies significantly outperform other schemes with or without the knowledge of short-term energy harvesting and channel fading patterns.

Motivation & Objective

• To address the challenge of maximizing long-term net bit rates in energy harvesting sensor networks with uncertain solar energy availability.
• To develop a node-specific, data-driven stochastic model of solar energy harvesting that reflects real-world variability from historical irradiance records.
• To design optimal transmission policies that adapt power levels, modulation schemes, and transmission activity based on dynamic battery states and channel fading.
• To evaluate the performance of these policies without requiring non-causal knowledge of future energy or channel conditions.

Proposed method

• Formulates the energy harvesting and transmission problem as a discounted Markov decision process (MDP) with state space defined by battery levels and hidden solar states.
• Uses historical solar irradiance data to stochastically quantize energy harvesting into discrete solar states, enabling data-driven modeling of energy arrival dynamics.
• Employs a mixed strategy to compute belief states over hidden solar states based on real-time irradiance observations, enabling adaptive policy decisions.
• Designs an on-off transmission policy with a threshold structure in battery state space, proven optimal under certain conditions.
• Introduces a composite policy that combines multiple transmission actions and evaluates performance across varying battery capacity and Doppler frequency.
• Uses computer simulations with real solar irradiance data from 2011–2012 to validate the framework and compare performance against benchmarks.

Experimental results

Research questions

• RQ1 How can a data-driven stochastic model of solar energy harvesting be constructed from real irradiance measurements to reflect node-specific energy availability?
• RQ2 What transmission policy structure maximizes long-term net bit rates under random energy arrivals and time-varying channel conditions?
• RQ3 Does a threshold-based on-off policy exist for energy-adaptive transmission, and what is its theoretical performance guarantee?
• RQ4 How do battery capacity, solar panel size, and channel dynamics (e.g., Doppler frequency) affect the achievable net bit rate?
• RQ5 Can the proposed policy outperform existing schemes—especially those relying on prediction or myopic decisions—without requiring future knowledge of energy or channel states?

Key findings

• The proposed on-off transmission policy achieves a maximum spectrum efficiency of approximately 0.6 bits/sec/Hz for QPSK and 1.2 bits/sec/Hz for 16QAM, significantly outperforming myopic and prediction-based policies.
• With a solar panel area of 8 cm² and 16 battery states, the average net bit rate reaches about 2.5×10⁵ bits/sec at 0 dB SNR and 0.05 normalized Doppler frequency, representing a 1.5× improvement over a 2-state battery system.
• The performance gap between the proposed on-off policy and the myopic policy widens with higher-order modulation (e.g., 16QAM), demonstrating greater exploitation of channel diversity.
• The composite policy shows that increasing battery capacity enhances net bit rate substantially, especially at low SNR, with gains being more pronounced under higher Doppler frequencies.
• The proposed method achieves superior performance compared to the t-TFR scheme and Myopic Policy I/II, even without non-causal knowledge of energy or channel states, due to its belief-based adaptation to real-time solar conditions.
• The theoretical analysis confirms that the optimal on-off policy exhibits a threshold structure in battery state space, enabling efficient and scalable implementation.