[Paper Review] A Reinforcement Learning-assisted Genetic Programming Algorithm for Team Formation Problem Considering Person-Job Matching

Yangyang Guo, Hao Wang+4 more|arXiv (Cornell University)|Apr 8, 2023

Metaheuristic Optimization Algorithms Research4 citations

TL;DR

This paper proposes a Reinforcement Learning-assisted Genetic Programming (RL-GP) algorithm to solve the team formation problem with person-job matching (TFP-PJM), integrating intuitionistic fuzzy numbers for matching scores and using RL to dynamically select from four population search modes to balance exploration and exploitation. The method achieves up to 10x faster learning time and superior team efficiency compared to baseline GP, heuristic, and metaheuristic algorithms.

ABSTRACT

An efficient team is essential for the company to successfully complete new projects. To solve the team formation problem considering person-job matching (TFP-PJM), a 0-1 integer programming model is constructed, which considers both person-job matching and team members' willingness to communicate on team efficiency, with the person-job matching score calculated using intuitionistic fuzzy numbers. Then, a reinforcement learning-assisted genetic programming algorithm (RL-GP) is proposed to enhance the quality of solutions. The RL-GP adopts the ensemble population strategies. Before the population evolution at each generation, the agent selects one from four population search modes according to the information obtained, thus realizing a sound balance of exploration and exploitation. In addition, surrogate models are used in the algorithm to evaluate the formation plans generated by individuals, which speeds up the algorithm learning process. Afterward, a series of comparison experiments are conducted to verify the overall performance of RL-GP and the effectiveness of the improved strategies within the algorithm. The hyper-heuristic rules obtained through efficient learning can be utilized as decision-making aids when forming project teams. This study reveals the advantages of reinforcement learning methods, ensemble strategies, and the surrogate model applied to the GP framework. The diversity and intelligent selection of search patterns along with fast adaptation evaluation, are distinct features that enable RL-GP to be deployed in real-world enterprise environments.

Motivation & Objective

• Address the challenge of forming high-efficiency project teams by integrating person-job matching and communication willingness.
• Overcome limitations of traditional metaheuristics in search efficiency and parameter sensitivity for the TFP-PJM problem.
• Develop a hyper-heuristic GP framework that adapts search strategies intelligently using reinforcement learning.
• Improve solution quality and convergence speed through ensemble population strategies and surrogate model-based evaluation.
• Provide decision-support heuristics for real-world team formation in enterprise settings using learned, generalizable rules.

Proposed method

• Formulate TFP-PJM as a 0-1 integer programming model with intuitionistic fuzzy numbers to quantify person-job matching scores.
• Design an RL-GP algorithm that uses a reinforcement learning agent to select one of four population search modes (e.g., exploration, exploitation) based on state information.
• Implement an ensemble population strategy where the RL agent dynamically balances exploration and exploitation across generations.
• Integrate a K-Nearest Neighbor (K-NN) surrogate model to accelerate fitness evaluation by predicting team efficiency without full computation.
• Update the surrogate model online during evolution to improve prediction accuracy and guide search more effectively.
• Evolve GP individuals that combine low-level rules into high-level heuristics for team formation, with structure complexity adapting to problem scale.

Experimental results

Research questions

• RQ1 Can reinforcement learning effectively guide the selection of search modes in a genetic programming framework for team formation?
• RQ2 How does the integration of a surrogate model impact the convergence speed and solution quality in solving TFP-PJM?
• RQ3 To what extent does the ensemble population strategy enhance the balance between exploration and exploitation in GP-based team formation?
• RQ4 How do the learned heuristic rules generalize across different team sizes and skill-to-position ratios?
• RQ5 Can the RL-GP algorithm outperform traditional GP, genetic algorithms, and variable neighborhood search in both solution quality and computational efficiency?

Key findings

• The RL-GP algorithm reduces learning time to between 1/5 and 1/10 of that required by other GP methods, significantly accelerating the rule discovery process.
• RL-GP-generated heuristics achieve higher team efficiency than both constructed heuristics and baseline metaheuristics like genetic algorithm and variable neighborhood search.
• The algorithm demonstrates robust performance across different team sizes (100 and 200 members), with well-performing heuristics showing structural similarities indicating stable learning behavior.
• The surrogate model effectively accelerates evaluation, enabling faster adaptation and improved convergence without sacrificing solution quality.
• Heuristics with moderate depth show better generalization, while deeper heuristics perform well on specific instances but lack broad applicability.
• The RL agent learns to favor similar search patterns across problem instances, indicating consistent strategy selection based on state information.