[Paper Review] Learning Optimal Routing for the Uplink in LPWANs Using Similarity-enhanced epsilon-greedy

Sergio Barrachina‐Muñoz, Boris Bellalta|arXiv (Cornell University)|May 22, 2017

IoT and Edge/Fog Computing6 references4 citations

TL;DR

This paper proposes a similarity-enhanced epsilon-greedy reinforcement learning approach to optimize uplink routing in LPWANs, reducing energy consumption by intelligently balancing exploration and exploitation. It demonstrates that leveraging path similarity significantly improves energy savings in medium-to-large LPWAN deployments when combined with sufficient initial exploration, outperforming standard epsilon-greedy and brute-force methods.

ABSTRACT

Despite being a relatively new communication technology, Low-Power Wide Area Networks (LPWANs) have shown their suitability to empower a major part of Internet of Things applications. Nonetheless, most LPWAN solutions are built on star topology (or single-hop) networks, often causing lifetime shortening in stations located far from the gateway. In this respect, recent studies show that multi-hop routing for uplink communications can reduce LPWANs' energy consumption significantly. However, it is a troublesome task to identify such energetically optimal routings through trial-and-error brute-force approaches because of time and, especially, energy consumption constraints. In this work we show the benefits of facing this exploration/exploitation problem by running centralized variations of the multi-arm bandit's epsilon-greedy, a well-known online decision-making method that combines best known action selection and knowledge expansion. Important energy savings are achieved when proper randomness parameters are set, which are often improved when conveniently applying similarity, a concept introduced in this work that allows harnessing the gathered knowledge by sporadically selecting unexplored routing combinations akin to the best known one.

Motivation & Objective

• To address the high energy cost of multi-hop routing discovery in LPWANs using centralized online learning.
• To improve upon standard epsilon-greedy by introducing a novel similarity concept to enhance knowledge transfer between similar routing paths.
• To identify optimal hyperparameters (ε and εs) that maximize energy savings across diverse LPWAN topologies.
• To evaluate the performance of the proposed method in realistic LPWAN scenarios using the DRESG framework.
• To determine when and under what conditions similarity-enhanced exploration provides measurable energy savings.

Proposed method

• Adapts the multi-armed bandit epsilon-greedy algorithm for centralized uplink routing optimization in LPWANs.
• Introduces a similarity metric to identify routing paths similar to the current best-known path, enabling informed exploration.
• Applies a quadratically decreasing update function for both ε and εs to balance early exploration and late exploitation.
• Uses the DRESG framework to simulate various LPWAN topologies with varying numbers of rings (R) and tree children ratios (c).
• Evaluates energy consumption across iterations, comparing standard epsilon-greedy with similarity-enhanced variants.
• Defines the similarity improvement ratio ρs(i) = ℰ(i)/ℰs(i) to quantify performance gains from similarity-based exploration.

Experimental results

Research questions

• RQ1 How does the performance of epsilon-greedy routing learning vary with different initial ε and εs settings in LPWANs?
• RQ2 In what LPWAN topologies does similarity-based exploration significantly reduce energy consumption compared to standard epsilon-greedy?
• RQ3 What is the impact of network size (number of rings R) and tree children ratio (c) on the effectiveness of similarity-enhanced exploration?
• RQ4 Can similarity-based exploration outperform brute-force routing discovery in terms of energy efficiency and convergence speed?
• RQ5 Under what conditions does similarity enhancement fail or degrade performance, and why?

Key findings

• For small networks (3–4 rings), similarity-enhanced epsilon-greedy does not improve performance and often worsens energy consumption compared to standard epsilon-greedy.
• In larger networks (e.g., 10–15 rings), similarity enhancement can significantly reduce energy consumption, especially when combined with high initial exploration (ε = 1).
• The combination {ε = 1, εs = 1} achieves the best performance in large networks like scenario M, reducing energy use through effective similarity-based exploration.
• When tree children ratio (c) increases, the gap between optimal and suboptimal paths widens, making early discovery of efficient routes more critical and amplifying the benefit of similarity.
• In line networks (c = 1), similarity enhancement is counterproductive due to high path dissimilarity, especially in large rings (e.g., scenarios I and L).
• The optimal setting of ε and εs is highly dependent on network parameters such as R and c, making generalization difficult without prior knowledge of topology.