[Paper Review] Generative Job Recommendations with Large Language Model

Zhi Zheng, Zhaopeng Qiu+4 more|arXiv (Cornell University)|Jul 5, 2023

Radiomics and Machine Learning in Medical Imaging15 citations

TL;DR

The paper introduces GIRL, a three-stage training framework that uses an LLM to generate personalized job descriptions from a CV, then trains a reward model and applies PPO-based reinforcement learning to align outputs with recruiter preferences, while also enhancing traditional discriminative job recommendations with generated JDs.

ABSTRACT

The rapid development of online recruitment services has encouraged the utilization of recommender systems to streamline the job seeking process. Predominantly, current job recommendations deploy either collaborative filtering or person-job matching strategies. However, these models tend to operate as "black-box" systems and lack the capacity to offer explainable guidance to job seekers. Moreover, conventional matching-based recommendation methods are limited to retrieving and ranking existing jobs in the database, restricting their potential as comprehensive career AI advisors. To this end, here we present GIRL (GeneratIve job Recommendation based on Large language models), a novel approach inspired by recent advancements in the field of Large Language Models (LLMs). We initially employ a Supervised Fine-Tuning (SFT) strategy to instruct the LLM-based generator in crafting suitable Job Descriptions (JDs) based on the Curriculum Vitae (CV) of a job seeker. Moreover, we propose to train a model which can evaluate the matching degree between CVs and JDs as a reward model, and we use Proximal Policy Optimization (PPO)-based Reinforcement Learning (RL) method to further fine-tine the generator. This aligns the generator with recruiter feedback, tailoring the output to better meet employer preferences. In particular, GIRL serves as a job seeker-centric generative model, providing job suggestions without the need of a candidate set. This capability also enhances the performance of existing job recommendation models by supplementing job seeking features with generated content. With extensive experiments on a large-scale real-world dataset, we demonstrate the substantial effectiveness of our approach. We believe that GIRL introduces a paradigm-shifting approach to job recommendation systems, fostering a more personalized and comprehensive job-seeking experience.

Motivation & Objective

• Motivate the need for explainable, generation-based job recommendations beyond traditional discriminative methods.
• Propose a three-step training pipeline (SFT, reward modeling, reinforcement learning) to train an LLM to generate job descriptions from CVs.
• Demonstrate how generated JDs can serve as explanations and improve downstream recommendation performance.
• Showcase that generation-enhanced recommendations can outperform baseline discriminative models, especially in cold-start scenarios.

Proposed method

• Formulate generative job recommendation as generating a JD J' from a CV C using an LLM-based generator G.
• Step 1: Supervised Fine-Tuning (SFT) with matched CV-JD pairs to teach G to generate appropriate JDs via a prompt template.
• Step 2: Reward Model Training (RMT) to predict recruiter-style matching between CVs and JDs using paired matched/mismatched data.
• Step 3: Reinforcement Learning from Recruiter Feedback (RLRF) using PPO to align G with recruiter preferences, with a joint reward incorporating U(C,J') and KL-divergence terms.
• Generation-enhanced recommendation: combine generated JDs with traditional encoders to improve ranking via augmented features (J', embeddings) in MLP or Dot product predictors.
• Key equations include: (1) SFT loss -log Pr(C|J,T,G); (2) Ranking loss L_rmt = log sigma(U(C,J^+) - U(C,J^−)); (3)-(8) PPO-based actor-critic updates with KL-divergence and advantage calculations.

Experimental results

Research questions

• RQ1 RQ1: Can the LLM-based generator produce high-quality JDs for job seekers?
• RQ2 RQ2: Do generated JDs enhance the performance of discriminative job recommendation models?
• RQ3 RQ3: Are the proposed SFT, reward modeling, and RL training steps effective?
• RQ4 RQ4: How do different generation settings (e.g., number of JDs) affect performance and cost?

Key findings

• GIRL and GIRL-SFT outperform baselines in generation quality when evaluated against ChatGPT-based criteria (level of detail, relevance, conciseness).
• In discriminative recommendation, GIRL (with RL) achieves higher AUC and lower LogLoss than baselines across MLP and Dot predictors (e.g., AUC 0.6476 with MLP; 0.6436 with Dot).
• RL-based fine-tuning (GIRL) yields improvements over SFT-only (GIRL-SFT) in both generation quality and recommendation performance.
• Using generated JDs for enhancement is especially beneficial under cold-start conditions, with larger gains noted for Dot-based predictors.
• Increasing the number of generated JDs can improve performance up to a point, after which computational costs rise and gains diminish.