[论文解读] Harnessing Large Vision and Language Models in Agriculture: A Review

Hongyan Zhu, Shuai Qin+4 more|arXiv (Cornell University)|Jul 29, 2024

Smart Agriculture and AI被引用 5

一句话总结

此评估综述了大型视觉、语言以及视觉-语言模型（LVLM/MLLM）如何应对农业任务——从害虫/疾病检测到土壤和种子质量，以及农民决策支持。

ABSTRACT

Large models can play important roles in many domains. Agriculture is another key factor affecting the lives of people around the world. It provides food, fabric, and coal for humanity. However, facing many challenges such as pests and diseases, soil degradation, global warming, and food security, how to steadily increase the yield in the agricultural sector is a problem that humans still need to solve. Large models can help farmers improve production efficiency and harvest by detecting a series of agricultural production tasks such as pests and diseases, soil quality, and seed quality. It can also help farmers make wise decisions through a variety of information, such as images, text, etc. Herein, we delve into the potential applications of large models in agriculture, from large language model (LLM) and large vision model (LVM) to large vision-language models (LVLM). After gaining a deeper understanding of multimodal large language models (MLLM), it can be recognized that problems such as agricultural image processing, agricultural question answering systems, and agricultural machine automation can all be solved by large models. Large models have great potential in the field of agriculture. We outline the current applications of agricultural large models, and aims to emphasize the importance of large models in the domain of agriculture. In the end, we envisage a future in which famers use MLLM to accomplish many tasks in agriculture, which can greatly improve agricultural production efficiency and yield.

研究动机与目标

• 探索 LVLM/MLLM 如何改变农业任务与决策。
• 对害虫/疾病检测、土壤与种子质量以及自动化等领域的当前与潜在应用进行分类。
• 识别在农业中部署大型模型的挑战、局限性及数据需求。
• 强调未来方向及对农民有利的多模态农业 AI 的应用前景。

提出的方法

• 综述在农业领域的大型语言模型（LLMs）、大型视觉模型（LVMs）和 LVLMs 的现有文献。
• 分析诸如农业图像处理、农业问答系统以及自动化等领域。
• 讨论多模态数据融合、模型架构及农业任务所需信息。
• 概述在农业情境中数据、可靠性与部署等实际考虑。

实验结果

研究问题

• RQ1 LVLMs 与 MLLMs 可以解决哪些农业任务？
• RQ2 在农业中部署大型模型的主要挑战与局限性是什么？
• RQ3 多模态模型如何改善农民决策和农场管理？
• RQ4 实现农业广泛应用需要哪些未来方向与要求？

主要发现

• 大型模型有潜力提高农业的生产效率和产量。
• LVLMs/MLLMs 可以支持农业图像处理、问答系统和自动化工作流程。
• 多模态信息（图像、文本等）可以提升农民决策和任务自动化。
• 挑战包括领域自适应、数据可用性、可靠性，以及在实际农业条件下的部署。
• 未来工作设想面向农民的 MLLMs 能实现更广泛的农业任务。