[Paper Review] Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applications

Saed Rezayi, Zhengliang Liu+9 more|arXiv (Cornell University)|Jun 20, 2023

Culinary Culture and TourismAgricultural and Biological Sciences3 citations

TL;DR

This paper introduces AgriBERT, a BERT-based language model pre-trained from scratch on agricultural text corpora, demonstrating superior performance in food-related NLP tasks such as semantic matching between food and nutrition databases and cuisine prediction. AgriBERT outperforms general and domain-specific models, achieving an F1 score of 79.63% in cuisine prediction, highlighting its effectiveness for agricultural NLP applications with external knowledge integration via FoodOn ontology.

ABSTRACT

This paper explores new frontiers in agricultural natural language processing by investigating the effectiveness of using food-related text corpora for pretraining transformer-based language models. In particular, we focus on the task of semantic matching, which involves establishing mappings between food descriptions and nutrition data. To accomplish this, we fine-tune a pre-trained transformer-based language model, AgriBERT, on this task, utilizing an external source of knowledge, such as the FoodOn ontology. To advance the field of agricultural NLP, we propose two new avenues of exploration: (1) utilizing GPT-based models as a baseline and (2) leveraging ChatGPT as an external source of knowledge. ChatGPT has shown to be a strong baseline in many NLP tasks, and we believe it has the potential to improve our model in the task of semantic matching and enhance our model's understanding of food-related concepts and relationships. Additionally, we experiment with other applications, such as cuisine prediction based on food ingredients, and expand the scope of our research to include other NLP tasks beyond semantic matching. Overall, this paper provides promising avenues for future research in this field, with potential implications for improving the performance of agricultural NLP applications.

Motivation & Objective

• To develop a domain-specific language model tailored for agricultural and food-related NLP tasks.
• To improve semantic matching between food descriptions and nutrition data using pre-trained language models.
• To evaluate the effectiveness of incorporating external knowledge from the FoodOn ontology in enhancing model performance.
• To explore the potential of GPT-based models and ChatGPT as baselines and knowledge sources in agricultural NLP.
• To extend the scope of AgriBERT beyond semantic matching to include tasks like cuisine prediction and ingredient-based classification.

Proposed method

• Pre-train a BERT-based model, AgriBERT, from scratch on a large corpus of agricultural literature, including 1.2 million paragraphs from the National Agricultural Library and 5.3 million from agricultural books and general literature.
• Fine-tune AgriBERT on semantic matching tasks by reformulating them as answer selection problems using food description and nutrition data pairs.
• Enhance input text with entities from the FoodOn knowledge graph, injecting domain-specific knowledge to improve contextual understanding.
• Evaluate AgriBERT against general-purpose (e.g., all-mpnet-base-v2) and other domain-specific models (e.g., agri-sentence-transformer) across multiple classifiers.
• Use zero-shot and few-shot prompting with ChatGPT to explore its utility as a baseline and external knowledge source for food-related NLP tasks.
• Apply AgriBERT to cuisine prediction by fine-tuning on recipe-text classification, comparing performance across multiple classifiers.

Figure 1: The overall framework of AgriBERT which is trained on agriculture literature from scratch. AgriBERT is evaluated on the answer selection task. The answer selection component has two inputs: a question and an answer, and before we input them into the framework, we add new entities to them f

Experimental results

Research questions

• RQ1 Can a BERT-based language model pre-trained on agricultural text outperform general and existing domain-specific models in semantic matching between food and nutrition databases?
• RQ2 How does injecting external knowledge from the FoodOn ontology affect the performance of AgriBERT in semantic matching tasks?
• RQ3 To what extent can GPT-based models like ChatGPT serve as effective baselines or knowledge sources for agricultural NLP tasks?
• RQ4 Does AgriBERT generalize well beyond semantic matching, particularly in tasks like cuisine prediction?
• RQ5 What is the impact of different pre-trained language models on the performance of downstream food-related NLP applications?

Key findings

• AgriBERT achieved the highest F1 score of 79.63% in cuisine prediction across all tested classifiers, outperforming both general-purpose and domain-specific models.
• The inclusion of external entities from the FoodOn knowledge graph improved the P@1 metric in semantic matching but reduced performance on mean average precision, indicating a trade-off in evaluation metrics.
• AgriBERT consistently outperformed all other models, including the state-of-the-art all-mpnet-base-v2, across all classifiers in cuisine prediction, demonstrating its robustness and domain-specific effectiveness.
• The TF-IDF baseline achieved an F1 score of 70.76% with MLP but performed poorly with other classifiers, indicating limited generalization across models.
• Agri-sentence-transformer achieved an F1 score of 65.87% with SVM, showing domain-specific adaptation helps but falls short of AgriBERT’s performance.
• ChatGPT demonstrated strong baseline performance, suggesting its potential as a knowledge source for future agricultural NLP applications.