[Paper Review] Agricultural Knowledge Management Using Smart Voice Messaging Systems: Combination of Physical and Human Sensors

Naoshi Uchihira, Masami Yoshida|arXiv (Cornell University)|Aug 9, 2020

Smart Agriculture and AI9 references4 citations

TL;DR

This paper proposes a smart voice messaging system that integrates physical sensors with human sensory inputs (sight, hearing, smell, taste, touch) to enhance agricultural knowledge management. Farmers use smartphones to record verbal observations of crop and equipment conditions, which are transcribed and analyzed to improve agricultural decision-making; field trials at a Hokkaido greenhouse farm demonstrated the system's effectiveness in capturing and sharing actionable knowledge.

ABSTRACT

The use of the Internet of Things (IoT) in agricultural knowledge management systems is one of the most promising approaches to increasing the efficiency of agriculture. However, the existing physical sensors in agriculture are limited for monitoring various changes in the characteristics of crops and may be expensive for the average farmer. We propose a combination of physical and human sensors (the five human senses). By using their own eyes, ears, noses, tongues, and fingers, farmers could check the various changes in the characteristics and conditions (colors of leaves, diseases, pests, faulty or malfunctioning equipment) of their crops and equipment, verbally describe their observations, and capture the descriptions with audio recording devices, such as smartphones. The voice recordings could be transcribed into text by web servers. The data captured by the physical and human sensors (voice messages) are analyzed by data and text mining to create and improve agricultural knowledge. An agricultural knowledge management system using physical and human sensors encourages to share and transfer knowledge among farmers for the purpose of improving the efficiency and productivity of agriculture. We applied one such agricultural knowledge management system (smart voice messaging system) to a greenhouse vegetable farm in Hokkaido. A qualitative analysis of accumulated voice messages and an interview with the farmer demonstrated the effectiveness of this system. The contributions of this study include a new and practical approach to an "agricultural Internet of Everything (IoE)" and evidence of its effectiveness as a result of our trial experiment at a real vegetable farm.

Motivation & Objective

• To address the limitations of physical sensors in monitoring diverse crop and equipment conditions in agriculture.
• To leverage farmers' sensory observations—sight, hearing, smell, taste, and touch—as a low-cost, practical complement to physical sensors.
• To develop a scalable, low-cost system for agricultural knowledge management using voice messaging and text mining.
• To evaluate the effectiveness of integrating human sensory data with physical sensor data in a real-world farming environment.
• To promote knowledge sharing and productivity improvements among farmers through a voice-based data collection and analysis system.

Proposed method

• Farmers use smartphones to record verbal descriptions of crop and equipment conditions using their five senses.
• Voice recordings are transmitted to web servers for automatic transcription into text using speech-to-text technology.
• Transcribed text is analyzed via data and text mining techniques to extract actionable agricultural knowledge.
• Physical sensors (e.g., temperature, humidity) are integrated with human-observed data to create a hybrid sensing system.
• The system enables real-time knowledge capture, storage, and retrieval for improved decision-making.
• A prototype system was deployed and tested in a greenhouse vegetable farm in Hokkaido, Japan, to evaluate usability and effectiveness.

Experimental results

Research questions

• RQ1 How can human sensory observations be effectively integrated with physical sensor data to improve agricultural knowledge management?
• RQ2 What is the feasibility and usability of using voice messaging as a data collection method in agricultural settings?
• RQ3 To what extent can voice-recorded farmer observations contribute to identifying crop diseases, pests, and equipment malfunctions?
• RQ4 How does the integration of human and physical sensor data enhance knowledge sharing and decision-making among farmers?
• RQ5 What are the practical challenges and benefits of deploying a voice-based agricultural knowledge system in real-world farming?

Key findings

• The smart voice messaging system successfully captured diverse agricultural observations, including visual signs of disease and pest infestations, through farmers' verbal reports.
• Voice recordings were accurately transcribed and analyzed, enabling the extraction of meaningful patterns and actionable insights.
• The system demonstrated practical usability in a real greenhouse farm setting, with farmers able to consistently record and share observations.
• Qualitative analysis of voice messages revealed rich, context-specific data that physical sensors alone could not capture.
• Farmer interviews confirmed the system's value in improving awareness of crop conditions and facilitating knowledge transfer.
• The integration of human sensory input with physical sensors created a more comprehensive and cost-effective monitoring solution.