[论文解读] Large Language Models for Patent Classification: Strengths, Trade-offs, and the Long Tail Effect

Lorenzo Emer, Marco Lippi+2 more|arXiv (Cornell University)|Jan 30, 2026

Intellectual Property and Patents被引用 0

一句话总结

论文比较基于编码器的分类器与开源权重的大语言模型在CPC专利分类中的表现，突显在准确性、效率与长尾覆盖方面的互补优势与权衡。

ABSTRACT

Patent classification into CPC codes underpins large scale analyses of technological change but remains challenging due to its hierarchical, multi label, and highly imbalanced structure. While pre Generative AI supervised encoder based models became the de facto standard for large scale patent classification, recent advances in large language models (LLMs) raise questions about whether they can provide complementary capabilities, particularly for rare or weakly represented technological categories. In this work, we perform a systematic comparison of encoder based classifiers (BERT, SciBERT, and PatentSBERTa) and open weight LLMs on a highly imbalanced benchmark dataset (USPTO 70k). We evaluate LLMs under zero shot, few shot, and retrieval augmented prompting, and further assess parameter efficient fine tuning of the best performing model. Our results show that encoder based models achieve higher aggregate performance, driven by strong results on frequent CPC subclasses, but struggle on rare ones. In contrast, LLMs achieve relatively higher performance on infrequent subclasses, often associated with early stage, cross domain, or weakly institutionalised technologies, particularly at higher hierarchical levels. These findings indicate that encoder based and LLM based approaches play complementary roles in patent classification. We additionally quantify inference time and energy consumption, showing that encoder based models are up to three orders of magnitude more efficient than LLMs. Overall, our results inform responsible patentometrics and technology mapping, and motivate hybrid classification approaches that combine encoder efficiency with the long tail coverage of LLMs under computational and environmental constraints.

研究动机与目标

• 推动大规模、分层、且标签不平衡的CPC专利自动分类。
• 评估在稀有或表示较弱的CPC子类中，LLM是否能补充基于编码器的模型。
• 评估编码器与LLM方法在推理效率与环境影响方面的差异。
• 探索平衡准确性、可扩展性与可持续性的混合策略。
• 在独立的专利数据集上提供外部验证以测试泛化性。

提出的方法

• 在USPTO-70k CPC分类任务上比较受监督的编码器基线（BERT、SciBERT、PatentSBERTa）与开源权重的LLMs的表现。
• 评估LLMs的零-shot、少样本及检索增强提示。
• 对表现最佳的LLM配置应用参数高效的LoRA微调。
• 使用CodeCarbon量化能耗与CO2排放，分析可持续性权衡。
• 在标签阶层下评估层次和宏/微平均指标，并通过自助法置信区间与统计检验进行跨标签分层比较。

Figure 1 : Experimental pipeline used to evaluate supervised encoder models and instruction-tuned LLMs on hierarchical, multi-label CPC classification. Approximately 70 000 USPTO patents are preprocessed and split chronologically into training (2006–2017), validation (2018), and test (2019) sets. En

实验结果

研究问题

• RQ1 与LLMs相比，编码器基模型在常见CPC子类上的表现如何？
• RQ2 LLMs在CPC层级的覆盖是否更均衡，尤其在稀有或新兴类别上？
• RQ3 在大规模使用时，使用LLMs与编码器模型的效率与能耗有何差异？
• RQ4 混合方法是否能够在鲁棒专利分类中结合两种范式的优势？
• RQ5 在外部数据集（如EPO专利）上不重新训练时，发现是否具有良好泛化性？

主要发现

• 编码器基模型在总体性能上更高，受常见CPC子类驱动。
• LLMs在不常见子类和更高层级的覆盖上表现相对更强。
• 与编码器模型相比，LLMs在计算与能耗方面成本显著更高。
• 没有单一方法占据统治地位；编码器在效率与可扩展性方面表现优越，LLMs提供互补的长尾覆盖。
• 领域自适应的编码器（如SciBERT）优于通用编码器，检索增强提示提升LLM在稀有类别上的召回率。
• 将编码器效率与LLM长尾能力结合的混合工作流在实际资源约束下具有动机。

(a) Frequency of CPC subclasses in the training set, ordered by decreasing frequency.