[Paper Review] OODIDA: On-board/Off-board Distributed Data Analytics for Connected Vehicles

Gregor Ulm, Emil Gustavsson+1 more|arXiv (Cornell University)|Feb 1, 2019

Distributed systems and fault tolerance22 references4 citations

TL;DR

OODIDA is a scalable, on-board/off-board distributed data analytics platform for connected vehicles that enables efficient task distribution and concurrent processing across resource-constrained edge devices. Built with Erlang/OTP for message passing and supporting language-agnostic endpoints, it handles large-scale data workloads effectively despite limited client resources.

ABSTRACT

Connected vehicles may produce gigabytes of data per hour, which makes centralized data processing impractical at the fleet level. In addition, there are the problems of distributing tasks to edge devices and processing them efficiently. Our solution to this problem is OODIDA (On-board/off-board Distributed Data Analytics), which is a platform that tackles both task distribution to connected vehicles as well as concurrent execution of large-scale tasks on arbitrary subsets of clients. Its message-passing infrastructure has been implemented in Erlang/OTP, while the end points are language-agnostic. OODIDA is highly scalable and able to process a significant volume of data on resource-constrained clients.

Motivation & Objective

• To address the challenge of processing massive data volumes generated by connected vehicles at scale.
• To enable efficient distribution and execution of data analytics tasks across heterogeneous, resource-constrained edge devices.
• To design a system that supports concurrent processing on arbitrary subsets of connected vehicles.
• To ensure high scalability and performance in real-world fleet-level deployments.
• To provide a flexible, language-agnostic architecture for integrating diverse data processing workloads.

Proposed method

• OODIDA employs a message-passing infrastructure implemented in Erlang/OTP to manage communication and coordination among vehicles and off-board servers.
• It enables task distribution to on-board devices by dynamically selecting subsets of connected vehicles based on availability and capability.
• The platform supports concurrent execution of analytics tasks across selected vehicles, leveraging their computational capacity.
• Endpoints are designed to be language-agnostic, allowing integration with diverse data processing pipelines.
• The system is architected for horizontal scaling, enabling it to handle large-scale data workloads across fleets.
• Resource constraints on vehicles are addressed through lightweight, efficient task execution and message handling.

Experimental results

Research questions

• RQ1 How can data analytics tasks be efficiently distributed across a fleet of connected vehicles with limited computational resources?
• RQ2 What architecture enables concurrent execution of large-scale analytics on arbitrary subsets of vehicles?
• RQ3 How can a distributed system maintain scalability and reliability when processing gigabytes of data per hour per vehicle?
• RQ4 What role does a language-agnostic interface play in enabling interoperability across diverse data processing components?
• RQ5 How does the use of Erlang/OTP enhance fault tolerance and message-passing efficiency in a vehicular edge environment?

Key findings

• OODIDA successfully enables distributed data analytics across fleets of connected vehicles, handling large-scale data workloads despite resource constraints.
• The platform demonstrates high scalability due to its message-passing architecture built on Erlang/OTP, which supports efficient concurrency and fault tolerance.
• By offloading processing to on-board devices, OODIDA reduces reliance on centralized data centers, improving system resilience and reducing latency.
• The language-agnostic design allows seamless integration of diverse data processing modules, enhancing extensibility and interoperability.
• OODIDA achieves efficient concurrent execution of analytics tasks on arbitrary subsets of vehicles, enabling flexible and dynamic workload distribution.
• The system is capable of processing significant data volumes on edge devices, validating its feasibility for real-world deployment in connected vehicle ecosystems.