Multi-Interest Sequential Recommendation Based on Prompt-Oriented BERT

Abstract

Existing sequential recommendation models primarily rely on item ID sequences, which ignore semantic information within user behaviors and diverse, dynamic interests. To address these challenges, this paper proposes a unified framework to model users' base and temporary interests with lower resource consumption. Our core method converts item titles, item genres, and user ratings in the order of interaction history into semi-structured natural-language prompts, thereby leveraging the semantic understanding capabilities of a pre-trained BERT model. Building on this, we design a multi-module fusion architecture: (1) a Mixture-of-Experts (MoE) network captures diverse user interests; (2) dedicated temporary interest and base preference modules model users' dynamic and stable tastes, respectively; (3) an adaptive fusion layer integrates these signals for precise prediction. Experimental results on benchmark datasets show that the proposed model outperforms all baseline models on Recall, and most models on NDCG. This study demonstrates that combining pre-trained language models with prompt engineering and multi-interest fusion modeling is a practical pathway toward building recommender systems.