1 Introduction↩︎

Large language models (LLMs) have recently enabled a new class of agentic recommender systems that model user preferences through natural-language reasoning and interaction [1]–[3]. Representative methods such as AgentCF [4] formulate recommendation as a collaborative process among language agents, where agents compare candidate items, generate rationales, and update their internal memories. This paradigm improves interpretability and flexibility by externalizing preference modeling into language-mediated reasoning rather than relying solely on gradient-based representation learning.

Despite their promise, existing LLM-based recommendation agents still suffer from two key limitations. First, most methods remain text-centric, constructing user and item memories mainly from textual histories, reviews, or metadata while underusing visual evidence such as product images [5]. This restricts their ability to model preferences in visually grounded domains such as fashion, electronics, and media products, where appearance, style, packaging, and other visual attributes often play an important role. Second, current memory update mechanisms are often coarse-grained. Agents typically revise their profiles through free-form reflection or direct history accumulation, which can introduce redundant descriptions, amplify spurious rationales, and cause preference drift over repeated interactions. These issues become more challenging when textual and visual signals must be jointly interpreted and selectively consolidated.

To address these challenges, we propose MMEACR, a Multimodal Memory-Enhanced Agent Collaboration framework for recommendation. MMEACR is motivated by the idea of Seeing and Reflecting. Seeing grounds agents in multimodal evidence by initializing and enriching item memories with both textual metadata and image-derived descriptions. Reflecting enables structured memory evolution by reinforcing aligned preference signals and correcting misleading ones after user–item interactions.

MMEACR contains two complementary tracks. The reasoning track uses cooperative User and Item Memory Agents to maintain persistent language memories. During interaction, an LLM-based agent compares candidate items, generates a rationale, and updates memories according to whether the predicted preference matches the observed feedback. To reduce noisy updates, MMEACR extracts preference-relevant attributes from a predefined semantic attribute space and uses them to guide both reinforcement and reflection. This helps preserve stable user interests while revising inaccurate or drifting memory descriptions.

The matching track complements language-based reasoning with dense multimodal representations. Since structured memory updates may filter out fine-grained details, MMEACR also maintains raw interaction narratives and item images for embedding-based matching. A multimodal embedding module encodes these signals into dense representations, capturing cross-modal semantics that may be difficult to express in concise textual memories. Finally, MMEACR combines the reasoning-based and embedding-based rankings through weighted Reciprocal Rank Fusion, allowing the final recommendation to benefit from both interpretable agent reasoning and fine-grained multimodal similarity.

Our contributions are summarized as follows:

• We propose MMEACR, a multimodal memory-enhanced agent collaboration framework that grounds LLM-based recommendation agents in both textual and visual evidence.

• We design an attribute-guided memory evolution mechanism that enables User and Item Memory Agents to reinforce aligned preferences and reflect on misaligned ones, reducing semantic noise and preference drift.

• We introduce a dual-track recommendation pipeline that combines interpretable language-agent reasoning with multimodal embedding-based matching through Reciprocal Rank Fusion, achieving strong performance across multiple domains.

2 Related Work↩︎

2.1 LLM-based Recommendation↩︎

Large language models (LLMs) have recently been explored for recommendation due to their strong abilities in semantic understanding, instruction following, and natural-language reasoning [6]–[8]. Early LLM-based recommenders mainly use frozen LLMs as zero-shot or few-shot rankers, where user histories and candidate items are converted into textual prompts for preference prediction [6]. Other studies further improve task adaptation through Chain-of-Thought reasoning, instruction tuning, or retrieval-augmented generation, enabling LLMs to better align textual semantics with collaborative signals [9]–[11]. More recently, agentic recommender systems have represented users and items as autonomous language agents that interact, exchange feedback, and update their memories during recommendation [4], [5]. These methods improve interpretability by making preference modeling explicit in language-based reasoning processes. Meanwhile, multimodal LLMs and multimodal embedding models have shown the potential to incorporate visual and textual signals for richer item understanding [12], [13]. However, most existing agentic recommenders still rely primarily on textual memories or directly inject multimodal information into prompts, which may lead to verbose contexts, redundant semantics, and unstable reasoning. In contrast, MMEACR adopts a dual-track design that separates structured agentic reasoning from dense multimodal matching, allowing the framework to use visual evidence without overloading the language-agent memory.

2.2 Memory in LLM Agents↩︎

Memory is essential for LLM agents to maintain consistency, accumulate experience, and support reflection across interactions [14]–[16]. Existing agent memory mechanisms range from short-term context buffers to long-term episodic stores, retrieval-based memories, and reflection-based memory management [17]–[19]. Although these mechanisms have been widely studied in general-purpose agent systems, applying them to recommendations remains challenging. User preferences are often dynamic, sparse, and attribute-dependent, while item descriptions may contain noisy or redundant information. Simply appending interaction histories or retrieving past episodes can therefore introduce irrelevant details, amplify spurious preference signals, and cause preference drift over time [20], [21]. Recent memory-enhanced recommender agents attempt to update user and item profiles through interaction feedback, but their updates are often coarse-grained and lack explicit constraints on which semantic attributes should be reinforced or corrected. MMEACR addresses this limitation with an attribute-guided memory evolution mechanism, where preference-relevant attributes are extracted from user-item comparisons and used to guide reinforcement and reflection. This design enables agents to consolidate stable preference signals while reducing semantic noise in long-term multimodal memory.

3 Problem Formulation↩︎

Let \(\mathcal{U}\) and \(\mathcal{I}\) denote the sets of users and items, respectively. For each user \(u \in \mathcal{U}\), we assume a chronologically ordered interaction history that reflects the user’s past preferences. Given a candidate item set \(\mathcal{C} = \{c_1, \dots, c_n\} \subseteq \mathcal{I}\), the goal is to generate a personalized ranking of these candidates according to the user’s preference. Formally, we define the recommendation function as: \[\hat{\mathcal{C}} = f_{\mathrm{LLM}}(u, \mathcal{C}),\] where \(\hat{\mathcal{C}}\) denotes the ranked candidate list produced by the LLM-based recommendation framework.

Memory in Agents. Following the agent-based recommendation paradigm [4], we model the collaborative recommendation process with two types of LLM-powered memory agents: a User Memory Agent (UA) and an Item Memory Agent (IA). Rather than relying on static textual profiles or fixed representation embeddings, MMEACR treats memories as persistent semantic states that can be updated through interactions. Specifically, the user memory \(M_u\) is an evolving textual narrative that summarizes the user’s long-term preferences, behavioral patterns, and fine-grained attribute tendencies. The item memory \(M_i\) describes item \(i\) by integrating its textual metadata with visual descriptions derived from product images. During recommendation and memory evolution, the User and Item Memory Agents collaborate through LLM-based reasoning over \(\{M_u\} \cup \{M_i\}\), enabling the framework to capture preference-relevant semantic signals and support reasoning-aware user–item interactions.

4 Methodology↩︎

We present MMEACR, a multimodal memory-enhanced agent collaboration framework for recommendation. As shown in Figure 2, MMEACR contains two complementary tracks: (1) a reasoning track, where User and Item Memory Agents maintain and update interpretable multimodal memories through LLM-based interaction; and (2) a matching track, where raw interaction narratives and item images are encoded into dense multimodal embeddings. The two tracks are finally combined through weighted Reciprocal Rank Fusion (RRF) to produce the final recommendation ranking.

4.1 Multimodal Agent Memory↩︎

4.1.1 Memory Initialization↩︎

MMEACR first initializes semantic memories for users and items. For each item \(i \in \mathcal{I}\), let \(T_i\) denote its textual title and \(\mathcal{V}_i=\{v_{i,1},\dots,v_{i,N_i}\}\) denote its associated images, where \(N_i \leq 5\). To convert visual evidence into language-readable semantics, we use a multimodal LLM to generate an image-grounded description: \[D_i = \mathrm{MLLM}(T_i, \mathcal{V}_i).\] The initial item memory is then constructed by integrating the title and visual description: \[M_i^{(0)} = \mathrm{LLM}(T_i, D_i).\] Here, \(M_i^{(0)}\) serves as a multimodal textual memory that summarizes the item’s key attributes. For each user \(u\), we initialize the user memory \(M_u^{(0)}\) with a domain-specific template, such as a general preference statement for music, electronics, or fashion products. In addition to structured memories, we maintain raw narratives \(H_u\) and \(H_i\) for users and items, which preserve unfiltered interaction histories and multimodal descriptions for embedding-based matching.

4.1.2 Attribute-Guided Memory Evolution↩︎

The reasoning track updates agent memories through contrastive user-item interactions. At each interaction step \(t\), we construct a triplet \((u,i^+,i^-)\), where \(i^+\) is the ground-truth preferred item and \(i^-\) is a sampled negative item. Given the current user memory \(M_u^{(t)}\) and item memories \(M_{i^+}^{(t)}\) and \(M_{i^-}^{(t)}\), the LLM-based interaction agent selects the item that better matches the user’s preference and generates a rationale: \[a^{(t)}, r^{(t)} = f_{\mathrm{dec}}\left(M_u^{(t)}, M_{i^+}^{(t)}, M_{i^-}^{(t)}\right),\] where \(a^{(t)} \in \{i^+, i^-\}\) is the selected item and \(r^{(t)}\) is the natural-language rationale. The correctness of the selection is determined by: \[y^{(t)} = \mathbb{I}\left[a^{(t)} = i^+\right].\]

To avoid updating memories with noisy free-form rationales, MMEACR introduces an attribute-guided preference extraction step. Let \(\mathcal{A}=\{a_1,\dots,a_K\}\) be a predefined semantic attribute space, where each attribute corresponds to a high-level preference factor such as style, functionality, portability, or visual appearance. Conditioned on the current memory, the contrastive item pair, and the generated rationale, the attribute extractor produces a structured preference signal: \[\mathbf{z}^{(t)} = f_{\mathrm{attr}}\left(M_u^{(t)}, T_{i^+}, T_{i^-}, r^{(t)}, \mathcal{A}\right),\] where \(\mathbf{z}^{(t)} \in \mathbb{R}^{K}\) encodes the attributes that explain the preference difference between \(i^+\) and \(i^-\).

The extracted attribute signal is then used to guide memory evolution. If the LLM selection is correct, MMEACR reinforces the current preference pattern; otherwise, it performs reflection to correct the misaligned memory: \[\small \begin{align} \mathbf{x}_u^{(t)} &= \left(M_u^{(t)}, M_{i^+}^{(t)}, M_{i^-}^{(t)}, \mathbf{z}^{(t)}, r^{(t)}\right), \\ M_u^{(t+1)} &= \begin{cases} f_{\mathrm{rein}}\!\left(\mathbf{x}_u^{(t)}\right), & y^{(t)}=1, \\ f_{\mathrm{refl}}\!\left(\mathbf{x}_u^{(t)}\right), & y^{(t)}=0. \end{cases} \end{align}\] This design allows the User Memory Agent to consolidate stable interests while revising inaccurate or drifting preference descriptions.

The Item Memory Agents are updated asynchronously for the interacted positive and negative items: \[\begin{align} M_{i^+}^{(t+1)} &= f_{\mathrm{pos}}\left(M_{i^+}^{(t)}, M_u^{(t)}, \mathbf{z}^{(t)}, y^{(t)}\right), \\ M_{i^-}^{(t+1)} &= f_{\mathrm{neg}}\left(M_{i^-}^{(t)}, M_u^{(t)}, \mathbf{z}^{(t)}, y^{(t)}\right). \end{align}\] Meanwhile, the raw user narrative is updated by appending the preferred item’s metadata and visual description: \[H_u^{(t+1)} = H_u^{(t)} \oplus [T_{i^+}; D_{i^+}],\] where \(\oplus\) denotes chronological concatenation. Similarly, raw item narratives are maintained for the interacted items. These raw narratives are not filtered by the attribute space, ensuring that fine-grained cross-modal information remains available for the embedding-based matching track.

4.1.3 Reasoning-Based Ranking↩︎

After iterative memory evolution, the reasoning track ranks candidate items using the evolved user and item memories. Given a candidate set \(\mathcal{C}\), the LLM-based ranker compares the user memory \(M_u\) with the candidate item memories \(\{M_c \mid c \in \mathcal{C}\}\) and produces a description-based ranking: \[\pi_{\mathrm{des}} = \mathrm{Rank}_{\mathrm{LLM}}\left(M_u, \{M_c \mid c \in \mathcal{C}\}\right).\] This ranking is interpretable because it is derived from explicit user and item memory descriptions.

4.2 Multimodal Embedding Memory↩︎

While the reasoning track provides interpretable preference reasoning, structured memory updates may filter out fine-grained visual or textual details. Therefore, MMEACR introduces a multimodal embedding memory to preserve raw cross-modal signals for dense matching.

4.2.1 Item Embedding Memory↩︎

For each item \(i\), we use a pretrained multimodal embedding model \(\mathrm{MEM}\) [13] to encode its title and images. The item embedding is computed by averaging image-title representations: \[\mathbf{e}_i = \frac{1}{N_i}\sum_{j=1}^{N_i} \mathrm{MEM}(T_i, v_{i,j}).\] This embedding captures both textual and visual item semantics.

4.2.2 User Embedding Memory↩︎

For each user \(u\), we construct the user embedding from the raw interaction narrative and the images of historically preferred items. Let \(\mathcal{P}_u\) denote the set of items preferred by user \(u\) in the interaction history. The user embedding is computed as: \[\mathbf{e}_u = \frac{1}{|\mathcal{P}_u|} \sum_{i \in \mathcal{P}_u} \mathrm{MEM}(H_u, \bar{v}_i),\] where \(\bar{v}_i\) denotes the representative visual content of item \(i\). Unlike the attribute-guided memory \(M_u\), the raw narrative \(H_u\) preserves complete historical descriptions, allowing the embedding track to capture detailed semantic and visual preference signals.

4.2.3 Embedding-Based Ranking↩︎

The embedding track ranks candidate items by cosine similarity: \[s_{\mathrm{emb}}(u,i) = \frac{\mathbf{e}_u^\top \mathbf{e}_i}{\|\mathbf{e}_u\| \|\mathbf{e}_i\|}.\] Sorting candidates by \(s_{\mathrm{emb}}(u,i)\) yields the embedding-based ranking \(\pi_{\mathrm{emb}}\).

4.3 Hybrid Ranking via Reciprocal Rank Fusion↩︎

The reasoning and matching tracks capture complementary signals. The reasoning track provides interpretable preference judgments based on evolved agent memories, while the embedding track captures fine-grained cross-modal similarity from raw narratives and images. To combine them, MMEACR adopts a weighted Reciprocal Rank Fusion strategy.

For each candidate item \(i\), we first compute its rank-based scores from the two tracks: \[\begin{align} S_{\mathrm{des}}(i) &= \frac{1}{k_{\mathrm{des}} + \mathrm{rank}_{\pi_{\mathrm{des}}}(i)}, \\ S_{\mathrm{emb}}(i) &= \frac{1}{k_{\mathrm{emb}} + \mathrm{rank}_{\pi_{\mathrm{emb}}}(i)}, \end{align}\] where \(\mathrm{rank}_{\pi_{\mathrm{des}}}(i)\) and \(\mathrm{rank}_{\pi_{\mathrm{emb}}}(i)\) denote the positions of item \(i\) in the reasoning-based and embedding-based rankings, respectively. The constants \(k_{\mathrm{des}}\) and \(k_{\mathrm{emb}}\) control the smoothness of rank contributions.

The final fusion score is: \[S_{\mathrm{RRF}}(i) = w_{\mathrm{des}} S_{\mathrm{des}}(i) + w_{\mathrm{emb}} S_{\mathrm{emb}}(i),\] where \(w_{\mathrm{des}}\) and \(w_{\mathrm{emb}}\) balance the contributions of the two tracks. The final recommendation list is obtained by sorting all candidate items in descending order of \(S_{\mathrm{RRF}}(i)\).

5 Experiments↩︎

Datasets. Following previous work [4], [22]–[24], we conduct experiments on three text-intensive subsets of the Amazon review dataset (CDs, Cell_phones, and Fashion).

Baselines. We evaluate our method against several representative baselines, including popularity-based Pop, text-matching BM25 [25], self-attention sequential recommender SASRec  [26] , LLM-based ranker LLMRank [27], embedding-similarity models LLMSeqSim [28] and MLLMSeqSim, as well as the agent-based AgentCF [4] and CoTAgent[9]. Specifically, CoTAgent incorporates zero-shot Chain-of-Thought (CoT) prompting to facilitate transparent decision-making and produce interpretable reasoning rationales for each recommendation. Pop ranks items by interaction frequency, while BM25 retrieves candidates using textual similarity to user histories. LLMRank uses GPT-4o-mini as a zero-shot ranker conditioned on sequential histories. LLMSeqSim constructs a session representation via LLM-generated item embeddings and retrieves top-k similar items, whereas MLLMSeqSim extends this with multimodal LLM embeddings. AgentCF treats users and items as autonomous LLM agents that interact and update memories to perform collaborative filtering. For our method MMEACR, we additionally include three variants. MMEACR-DES uses the user’s and item’s latest memories with a 1-positive-9-negative candidate set, prompting the LLM to return a ranking for computing NDCG and MMR. MMEACR-EMB fuses the user’s five recent item-image embeddings with the latest memory via GME, combines item image–title embeddings, and ranks candidates using cosine similarity. MMEACR-RRF merges the rankings from MMEACR-DES and MMEACR-EMB using the RRF formula.

5pt

Evaluation Metrics. We adopt a leave-one-out strategy  [26]: for each user, the last interaction is used for testing and the second-to-last for validation. Following previous works [4], each evaluation case includes one positive item and 9 randomly sampled negatives from the same domain.

Quantative Analysis. Table 1 reports the overall comparison across three domains. Our proposed MMEACR-RRF consistently delivers the strongest or highly competitive performance across datasets and metrics. On CDs, our method improves N@1 by 20.59% and MRR by 1.89% over the strongest baseline, while maintaining competitive performance on other metrics. The slightly lower NDCG@5 compared with CoTAgent may be attributed to the relatively low complexity of the CDs dataset, where explicit CoT reasoning is already sufficient to capture user preferences and rank relevant items effectively, leaving limited room for additional gains from attribute-guided reasoning and multimodal embedding fusion. On Cell_Phones, the improvements are consistent across all metrics, with gains of 14.3% on N@1 and 21.26% on N@5, demonstrating robust ranking quality in more attribute-diverse scenarios. Notably, on the visually intensive Fashion domain, our approach achieves substantial gains, improving N@1, N@5, and MRR by 45.45%, 23.33%, and 27.14%, respectively, highlighting the benefit of multimodal memory modeling in visually grounded recommendation.

Moreover, while both single-branch variants (MMEACR-DES and MMEACR-EMB) already achieve competitive performance, the fused model consistently yields further improvements, indicating that LLM-based reasoning and multimodal embedding similarity provide complementary signals. Compared with prior agent-based methods such as AgentCF and CoTAgent, our framework demonstrates consistent advantages across domains, validating the effectiveness of structured multimodal memory and reflective agent collaboration. To evaluate the computational efficiency of our proposed framework, we compare the inference time of MMEACR against the representative agent-based baseline, AgentCF, across three datasets. As illustrated in Figure 4, MMEACR consistently achieves lower inference latency than AgentCF in all evaluated domains. Specifically, MMEACR reduces inference time by 6.15% (from 70.12s to 65.81s) on CDs, 16.27% (from 132.52s to 110.96s) on Cell_Phones, and 15.11% (from 121.25s to 102.93s) on the Fashion dataset. These results demonstrate that while MMEACR incorporates structured multimodal memory and reflective collaboration to achieve superior ranking performance (as shown in Table 1), it simultaneously optimizes the inference pipeline, proving its high efficiency and practicality for real-world recommendation scenarios.

Qualitative Analysis. As shown in Figure 3, while AgentCF generates coarse-grained and repetitive interest summaries (e.g., general “jazz” or “phone accessories”), MMEACR precisely captures fine-grained, specific user preferences (highlighted in red) such as a “cohesive and relaxing mood” or “heavy-duty protection with stylish design.” Furthermore, while AgentCF is vulnerable to interaction noise and suffers from preference drift by Round-5, MMEACR filters cross-modal redundancy via attribute-guided reflection, anchoring user profiles onto stable semantic blocks. This qualitative alignment corroborates our quantitative superiority, visually demonstrating that MMEACR achieves precise long-term profiling through attribute-level memory refinement.

Ablation Study. To evaluate the contribution of each component in our MMEACR framework, we conduct ablation studies by comparing the full Multi-AgentCF model (with both User Agent and Item Agent) against three simplified variants: w/o Auto. Interaction, which removes the automated interaction mechanism between agents and disables iterative mutual refinement; w/o User Agent, which excludes the User Agent and relies solely on item-side modeling; and w/o Item Agent, which removes the Item Agent and retains only user-centric reasoning. Table 3 reports results on both cell_phones and fashion domains. The full Multi-AgentCF consistently achieves the best performance across most metrics, demonstrating the effectiveness of jointly modeling user and item agents within an interactive framework. In contrast, removing either agent leads to clear performance degradation, indicating that both user-side preference modeling and item-side contextual reasoning are essential for accurate ranking. Notably, the largest drop is observed when Auto. Interaction is removed, highlighting the importance of iterative inter-agent communication for refining memory and aligning preferences. These results verify that each component contributes to the overall performance and that their synergistic interaction is crucial for optimal recommendation quality.

Effectiveness of Iterative Memory Evolution. To investigate how iterative memory refinement enhances recommendation performance, we analyze the evolution of key metrics throughout the optimization process on the Fashion dataset. Specifically, we report results at three consecutive optimization steps and the final testing phase to capture how progressive agent collaboration affects ranking quality over time. As illustrated in Figure 5, all evaluation metrics, including \(N@1\), \(N@5\), \(N@10\), and \(MRR\), exhibit a consistent upward trend as iterative interaction between the user and item agents proceeds. In particular, the \(N@10\) score steadily increases from \(0.4463\) at Step 1 to \(0.5708\) at the final stage, while \(MRR\) improves from \(0.2833\) to \(0.4407\), with similar gains observed for \(N@1\) and \(N@5\), indicating consistent improvements across different ranking depths. These results suggest that iterative memory refinement enables continuous correction and enrichment of both user preference modeling and item contextual understanding, allowing the system to progressively accumulate more reliable reasoning traces and better align multimodal representations. Overall, the observed performance trajectory validates the effectiveness of our iterative update strategy and highlights the importance of continuous agent collaboration in improving recommendation quality.

6 Conclusion↩︎

In this paper, we propose MMEACR, a multimodal memory-enhanced agent collaboration framework for recommendation that overcomes modal-lacking limitations in text-only LLM-based systems by jointly leveraging visual and textual features and enabling dynamic memory evolution through iterative agent interactions. By incorporating a dedicated multimodal embedding memory and RRF-based ranking fusion, MMEACR produces more accurate and interpretable recommendations. Extensive experiments on real-world datasets show that MMEACR surpasses strong baselines, particularly in capturing nuanced user preferences from multimodal signals, and ablation studies further validate the contribution of each individual component. In addition, qualitative analyses demonstrate that the proposed agent collaboration mechanism can generate more consistent and preference-aligned reasoning across interaction steps. Overall, our results highlight the effectiveness of integrating multimodal memory with iterative agent collaboration for improving recommendation quality in LLM-based systems.

Limitations↩︎

While MMEACR demonstrates strong performance and efficient inference, one limitation is the long-term scalability of the continuous memory evolution. As user interactions scale over extended periods, the iterative accumulation in the multimodal embedding memory could potentially increase storage and retrieval costs. Although this is mitigated by our RRF-based ranking fusion in current benchmarks, optimizing the balance between memory capacity and lifelong efficiency remains a promising direction for our future work.

7 Computational Cost and Efficiency↩︎

Due to the high cost of LLM API calls, we sample 100 users and their historical interactions per dataset, leading to about 500 interaction steps across 5 training rounds. We use a tiered model strategy, where GPT-4o [29] is applied for memory initialization, interaction simulation, and inference-time ranking, balancing reasoning quality with efficiency and supporting asynchronous batch processing (batch size 4 for training, 5 for evaluation).

At inference, description-based ranking (DES) prompts GPT-4o with the user’s natural-language memory profile and 10 candidate item descriptions to directly output a ranked list, which is mapped back to item IDs using fuzzy matching. In parallel, embedding-based ranking (EMB) encodes user memory and multimodal item content into dense vectors and ranks candidates via cosine similarity without LLM inference. The final result is obtained by fusing DES and EMB rankings using Reciprocal Rank Fusion, with a fallback to EMB-only when DES outputs are invalid.

Overall, inference cost is dominated by a single LLM call per user, and full evaluation remains efficient, completing within about 100 seconds per dataset and showing a 6–16% speedup over AgentCF due to the streamlined memory design and shorter prompts.

8 Algorithm↩︎

9 Prompt Templates↩︎

References↩︎