[Paper Review] Large Language Models for Patent Classification: Strengths, Trade-offs, and the Long Tail Effect

Lorenzo Emer, Marco Lippi+2 more|arXiv (Cornell University)|Jan 30, 2026

Intellectual Property and Patents0 citations

TL;DR

The paper compares encoder-based classifiers with open-weight LLMs for CPC patent classification, highlighting complementary strengths and trade-offs in accuracy, efficiency, and long-tail coverage.

ABSTRACT

Patent classification into CPC codes underpins large scale analyses of technological change but remains challenging due to its hierarchical, multi label, and highly imbalanced structure. While pre Generative AI supervised encoder based models became the de facto standard for large scale patent classification, recent advances in large language models (LLMs) raise questions about whether they can provide complementary capabilities, particularly for rare or weakly represented technological categories. In this work, we perform a systematic comparison of encoder based classifiers (BERT, SciBERT, and PatentSBERTa) and open weight LLMs on a highly imbalanced benchmark dataset (USPTO 70k). We evaluate LLMs under zero shot, few shot, and retrieval augmented prompting, and further assess parameter efficient fine tuning of the best performing model. Our results show that encoder based models achieve higher aggregate performance, driven by strong results on frequent CPC subclasses, but struggle on rare ones. In contrast, LLMs achieve relatively higher performance on infrequent subclasses, often associated with early stage, cross domain, or weakly institutionalised technologies, particularly at higher hierarchical levels. These findings indicate that encoder based and LLM based approaches play complementary roles in patent classification. We additionally quantify inference time and energy consumption, showing that encoder based models are up to three orders of magnitude more efficient than LLMs. Overall, our results inform responsible patentometrics and technology mapping, and motivate hybrid classification approaches that combine encoder efficiency with the long tail coverage of LLMs under computational and environmental constraints.

Motivation & Objective

• Motivate automated CPC patent classification for large-scale, hierarchical, and imbalanced label spaces.
• Assess whether LLMs can complement encoder-based models for rare or weakly represented CPC subclasses.
• Evaluate inference efficiency and environmental impact of encoder vs. LLM approaches.
• Investigate potential hybrid strategies that balance accuracy, scalability, and sustainability.
• Provide external validation on an independent patent dataset to test generalizability.

Proposed method

• Compare supervised encoder baselines (BERT, SciBERT, PatentSBERTa) and open-weight LLMs on the USPTO-70k CPC classification task.
• Evaluate zero-shot, few-shot, and retrieval-augmented prompting for LLMs.
• Apply parameter-efficient LoRA fine-tuning to the best-performing LLM configuration.
• Quantify energy usage and CO2 emissions using CodeCarbon to analyze sustainability trade-offs.
• Assess hierarchical and macro/micro-averaged metrics with bootstrap confidence intervals and statistical testing across label strata.

Figure 1 : Experimental pipeline used to evaluate supervised encoder models and instruction-tuned LLMs on hierarchical, multi-label CPC classification. Approximately 70 000 USPTO patents are preprocessed and split chronologically into training (2006–2017), validation (2018), and test (2019) sets. En

Experimental results

Research questions

• RQ1 How do encoder-based models perform on frequent CPC subclasses compared with LLMs?
• RQ2 Do LLMs provide more balanced coverage across the CPC hierarchy, especially for rare or emerging categories?
• RQ3 What are the efficiency and energy implications of using LLMs versus encoder models at scale?
• RQ4 Can a hybrid approach leverage the strengths of both paradigms for robust patent classification?
• RQ5 How well do findings generalize to an external dataset (e.g., EPO patents) without retraining?

Key findings

• Encoder-based models deliver higher aggregate performance driven by frequent CPC subclasses.
• LLMs show relatively stronger performance on infrequent subclasses and at higher hierarchical levels.
• LLMs incur substantially higher computational and energy costs compared with encoder models.
• No single approach dominates; encoders excel in efficiency and scalability, while LLMs offer complementary long-tail coverage.
• Domain-adapted encoders (e.g., SciBERT) outperform generic encoders, and retrieval-augmented prompting improves LLM recall on rare classes.
• Hybrid workflows that combine encoder efficiency with LLM long-tail capabilities are motivated under practical resource constraints.

(a) Frequency of CPC subclasses in the training set, ordered by decreasing frequency.