1 Introduction↩︎

Large language model (LLM) agents are increasingly expected to support long-term personalized interaction, where useful knowledge must be accumulated across extended dialogue histories and reused in later conversations [1]–[3]. Recent memory-augmented systems have made substantial progress in reducing the cost of long-context inputs by extracting, compressing, and organizing historical interactions into external memory banks [4]–[10]. As these systems mature, a central challenge moves from memory construction alone to memory use at inference time: given a user query, the agent must retrieve evidence that is not only semantically relevant to the query, but also appropriate for the user behind the query [11]. In long-term dialogue, different users accumulate distinct memories that reflect their relationships, topical interests, and life contexts [12], [13]. The same query may therefore correspond to different information needs depending on who is asking. As illustrated in Figure 1, personalized memory retrieval requires going beyond surface query matching: the retriever must identify evidence that is jointly relevant to the query and to the user’s long-term profile.

Existing memory systems have not fully addressed this user-conditioned retrieval problem. Most work focuses on memory construction and management, including extraction, compression, hierarchical organization, graph-based indexing, and read-write operations [4]–[6]. These methods improve the coverage and efficiency of stored memories, but retrieval is often treated as a fixed downstream step based on semantic matching or heuristic scoring [14]–[16]. Personalized dialogue systems, on the other hand, model user-specific information [17]–[20], but usually focus on constructing personalized memory content or using profiles during generation rather than injecting user-level priors into retrieval ranking. As a result, current memory pipelines may retrieve memories that match the query surface while failing to prioritize evidence that is more informative under the user’s stable attributes, preferences, and relationships.

To address these issues, we propose Profile-guided Personalized Retrieval Optimization (PP RO), a retrieval-centric framework for personalized long-term conversational memory. PPRO first constructs a hierarchical memory representation from dialogue histories. It extracts episodic memories as fine-grained factual statements, aggregates related episodic memories into semantic memories, and summarizes stable user attributes, preferences, and relationships into a user profile. During inference, PPRO performs dual-path retrieval over both episodic and semantic memories. More importantly, the user profile is not only provided to the answer model as context, but also injected into the retrieval score as an embedding-level personalized prior. This design allows the retriever to rank memories by considering both query relevance and user-level relevance, making memory retrieval more sensitive to personalized information needs.

PPRO further optimizes retrieval through a trainable query rewriter. Because retrieving discrete memory items is non-differentiable, gradients cannot flow from answer quality back to the retriever. To bypass this, we train a query rewriter with Group Relative Policy Optimization (GRPO) [21], [22]: multiple rewritten queries are sampled and rewarded based on both evidence retrieval quality and final answer quality, while the memory banks, embedding model, and answer model remain frozen.

We evaluate PPRO on LoCoMo [1] and LongMemEval-S [23], two benchmarks for long-term conversational question answering. PPRO consistently outperforms both training-free memory systems and training-based baselines across both benchmarks, and ablation studies confirm the contribution of user-profile guidance, dual-path retrieval, retrieval dimensions, and GRPO-based query rewriting. The results show that personalized retrieval optimization is an effective complement to memory construction. Overall, the contributions of our work are threefold:

• We formulate personalized memory retrieval as a key problem in long-term conversational agents, emphasizing that retrieved evidence should be relevant to both the input query and the target user.

• We propose PPRO, combining profile-guided retrieval and GRPO-based query rewriting for personalized memory recall.

• Experiments on LoCoMo and LongMemEval-S show consistent gains over both training-free and training-based baselines.

2 Related Work↩︎

2.1 Long-Term Conversational Memory↩︎

As the field progresses, research on long-term conversational memory has evolved from memory construction and organization, to retrieval mechanism design, and more recently to learning-based optimization [24], [25].

Memory Construction and Organization. Effective memory organization is fundamental to long-term conversational agents. MemGPT [2] introduces OS-inspired virtual context management. A-Mem [5] employs Zettelkasten-inspired dynamic indexing, Mem0 [4] builds graph-based memory banks, and MemoryOS [7] proposes a multi-level memory operating system. SimpleMem [6] achieves strong performance through semantic lossless compression with multi-view indexing.

Memory Retrieval Mechanisms. Beyond better memory organization, how to accurately retrieve relevant memories given a query is equally critical. HippoRAG [14] draws from hippocampal indexing theory for multi-hop retrieval. THEANINE [26] retrieves via temporal-causal graph traversal, SynapticRAG [15] applies biologically-inspired synaptic propagation for temporal scoring, Zep [27] leverages temporal knowledge graphs, and CDMem [28] applies context-dependent indexing conditioned on task state.

RL-Based Memory Optimization. More recently, reinforcement learning has been applied to optimize memory systems end-to-end. Memory-R1 [29] trains policies for memory CRUD operations via GRPO. MEM1 [30] optimizes the think-search-answer loop, MemAgent [31] trains segmented reading for long-context compression, and Mem-T [32] employs memory-of-thought augmented GRPO. However, RL in these methods targets memory management or reasoning without being tightly coupled with the retrieval process. Our work unifies reinforcement learning with user-aware personalized retrieval, optimizing the retrieval process with reward signals derived from both evidence retrieval quality and answer quality.

2.2 User-Aware Retrieval↩︎

User-aware retrieval focuses on leveraging user-specific information to improve retrieval relevance. In recommendation, recent work enhances user-aware retrieval through personalized representations and preference modeling [33]–[37]. In conversation, PRIME [17] maps cognitive dual-memory theory onto LLM personalization, RMM [18] introduces reflective memory management with RL-refined retrieval, InsideOut [38] evolves user-centric core memory trees, and EMG-RAG [39] crafts personalized agents through editable memory graphs. Our work injects user profile embeddings directly into the retrieval scoring function as a personalized ranking prior, making user-aware retrieval end-to-end optimizable.

3 Methodology↩︎

We present Profile-guided Personalized Retrieval Optimization (PPRO), a retrieval-centric framework for personalized long-term conversational memory. As shown in Figure 2, PPRO constructs episodic memories, semantic memories, and user profiles from dialogue histories, and then uses the profile as a personalized prior to guide dual-path retrieval. A GRPO-trained query rewriter further optimizes retrieval using evidence retrieval quality and answer quality as feedback.

3.1 Problem Formulation↩︎

Given a target user \(u\), we assume access to the user’s long-term dialogue history \(D_u=\{d_1,\ldots,d_T\}\), where each dialogue \(d_t\) contains a sequence of timestamped utterances from multiple speakers. Given a history-grounded query \(q\), the task is to generate an answer \(y\) that matches the reference answer \(y^*\).

We focus on personalized memory retrieval as the key intermediate step. PPRO represents \(D_u\) with an episodic memory bank \(\mathcal{M}^E_u\), a semantic memory bank \(\mathcal{M}^S_u\), and a user profile \(\mathcal{P}_u\). At inference time, the retriever selects memories from the episodic and semantic memory banks that are both query-relevant and user-relevant. These retrieved memories, together with the user profile, are then provided to a frozen answer model \(\mathcal{A}\) to generate the final answer.

3.2 Offline Memory Construction↩︎

The offline phase converts raw dialogue histories into persistent user-specific memory resources. For each target user, PPRO builds three complementary components: episodic memories for fine-grained factual evidence, semantic memories for cross-dialogue aggregation, and a user profile for stable personalized information. We use the same instruction-following language model \(\mathcal{G}\) across all offline construction stages, with task-specific prompts.

Episodic Memory. For each dialogue \(d_t\), \(\mathcal{G}\) converts dialogue utterances into a set of atomic episodic memories. Each memory unit \(e\) is a factual textual statement paired with retrieval dimensions: \[\mathcal{M}_{u,t}^{E} = \mathcal{G}(d_t,u), \quad e=(c,\delta)\in \mathcal{M}_{u,t}^{E},\] where \(c\) is the textual content and \(\delta\) denotes the retrieval dimensions attached to the memory, including tense, subjectivity, and certainty. These retrieval dimensions provide structured cues for consistent memory extraction and are later used as dimension-aware matching signals in retrieval scoring. The full episodic memory bank for user \(u\) is \(\mathcal{M}^E_u = \bigcup_{t=1}^{T} \mathcal{M}^E_{u,t}\), providing fine-grained factual evidence for downstream retrieval.

Semantic Memory. Episodic memories are atomic and dialogue-local, making it difficult to answer queries that require cross-dialogue synthesis. To capture higher-level patterns, PPRO introduces a semantic memory bank \(\mathcal{M}_u^S\) by clustering episodic memories based on cosine similarity: \[\mathrm{sim}(e_i,e_j)=\cos(\mathcal{E}(e_i),\mathcal{E}(e_j)),\] where \(\mathcal{E}\) denotes the embedding model. Episodic memories whose similarity exceeds a threshold \(\tau_c\) are grouped into clusters \(C_g\). For each resulting cluster \(C_g\), the language model \(\mathcal{G}\) summarizes its episodic memories into a semantic memory unit: \[s = \mathcal{G}(C_g), \quad s \in \mathcal{M}_u^S.\]

User Profile. While episodic memories preserve local facts and semantic memories capture cross-dialogue patterns, many personalized queries benefit from a high-level representation of a user’s stable characteristics, including attributes, preferences, and relationships. Inspired by personalized recommendation in recommender systems [40], we propose aggregating user-level profiles to enhance memory retrieval during the online stage. PPRO constructs a textual user profile \(\mathcal{P}_u\) by summarizing the user’s semantic memory bank: \[\mathcal{P}_u = \mathcal{G}(\mathcal{M}^S_u), \quad \mathbf{p}_u = \mathcal{E}(\mathcal{P}_u),\] where \(\mathcal{G}\) summarizes long-term preferences, occupations, relationships, habits, and recurring interests, and \(\mathcal{E}\) is the shared embedding model. The profile is stored both as text \(\mathcal{P}_u\) and as an embedding vector \(\mathbf{p}_u\), which later serves as a personalized prior for online retrieval.

3.3 Online Inference and Optimization↩︎

At the online stage, PPRO first rewrites the input query into a retrieval-oriented form, then retrieves complementary evidence from episodic and semantic memory banks, and finally generates an answer from the assembled memory context. Personalization is introduced by injecting the user profile as a prior into episodic memory scoring, while the query rewriter is further optimized with GRPO using retrieval and answer-quality feedback. The complete procedure is summarized in Algorithm 6 in the Appendix.

Retrieval-Oriented Query Rewriting. The rewriter model \(f_\theta\) transforms the original query \(q\), conditioned on dialogue context \(C\), into a retrieval-oriented query \(q'\) and retrieval dimensions \(\delta_q\): \[(q', \delta_q) = f_\theta(q, C).\] The rewritten query \(q'\) is used for memory retrieval, while \(\delta_q\) provides dimension-level signals that complement semantic similarity in episodic retrieval.

Dual-path Memory Retrieval. PPRO retrieves from episodic and semantic memory banks in parallel. The episodic path focuses on fine-grained factual evidence, while the semantic path provides compact cross-dialogue summaries.

Episodic Retrieval. PPRO scores each candidate \(e\) by fusing query relevance with a profile-based personalized prior, augmented by a retrieval-dimension boost: \[\sigma(e) = \lambda\, \sigma_q(e) + (1-\lambda)\, \sigma_p(e) + b_\delta(e),\] where \(\sigma_q(e)=\cos(\mathcal{E}(q'), \mathcal{E}(e))\) measures query-memory relevance, \(\sigma_p(e)=\cos(\mathbf{p}_u, \mathcal{E}(e))\) captures user-profile relevance via the profile embedding \(\mathbf{p}_u\), and \(b_\delta(e)\) is the number of matching retrieval dimensions between \(\delta_q\) and \(\delta_e\) multiplied by a fixed bonus (0.05 per match across three dimensions: tense, subjectivity, and certainty). The hyperparameter \(\lambda\) controls the trade-off between query-driven and profile-driven retrieval.

Semantic Retrieval. In parallel, PPRO retrieves from \(\mathcal{M}^S_u\) using the same rewritten query \(q'\). Because semantic memories are already aggregated from user-specific episodic memories, they provide compact high-level context and are ranked directly by query similarity. Semantic memories are therefore ranked by standard cosine similarity \(\cos(\mathcal{E}(q'), \mathcal{E}(s))\).

Answer Generation. The retrieved memories are concatenated into a unified memory context \(\mathcal{C}_m = [\mathcal{P}_u; \mathcal{R}^E; \mathcal{R}^S]\), and the answer model \(\mathcal{A}\) generates the answer: \[y = \mathcal{A}(q, \mathcal{C}_m).\]

3.4 GRPO-based Optimization↩︎

The quality of memory-augmented QA depends heavily on whether the rewritten query can retrieve the correct evidence. Following recent work that uses multi-aspect feedback to train query rewriters [41]–[43], we optimize the rewriter \(f_\theta\) through the online inference pipeline using Group Relative Policy Optimization (GRPO) [21]. Instead of optimizing rewriting quality in isolation, \(f_\theta\) is trained with feedback from both evidence retrieval quality and final answer quality.

Policy and Environment. The rewriter \(f_\theta\) is treated as the policy \(\pi_\theta\). For each training data, the state is the original query \(q\) with its dialogue context \(C\), and the policy samples a group of \(N\) actions, corresponding to rewritten queries \(\{q'_1,\ldots,q'_N\}\) from \(\pi_\theta(\cdot \mid q, C)\). For each sampled rewrite \(q'_i\), PPRO executes the online inference pipeline to obtain retrieved episodic memories \(\mathcal{R}^E_i\), semantic memories \(\mathcal{R}^S_i\), and a generated answer \(y_i\).

Reward Design. The reward combines a retrieval reward and an answer reward: \[R_i = \alpha \widetilde{R}_{ans,i} + (1-\alpha) \widetilde{R}_{ret,i},\] where \(\alpha\) controls the trade-off between the two signals. The retrieval reward \(R_{ret,i}\) is the F1 score between the source turns of retrieved episodic memories and ground-truth evidence annotations. The answer reward \(R_{ans,i} = \mathrm{BLEU\text{-}1}(y_i, y^*)\) measures unigram precision between the generated answer and the reference [44]. Both rewards are normalized via EMA-based adaptive scaling before being combined.

Group-relative Advantage and Optimization. After executing the pipeline for each of the \(N\) rewrites, the group-relative advantage is: \[\hat{A}_i = \frac{R_i - \mathrm{mean}(\{R_j\}_{j=1}^{N})}{\mathrm{std}(\{R_j\}_{j=1}^{N}) + \epsilon}.\] This eliminates the need for a separately trained critic and directly compares alternative rewrites for the same query. The rewriter is then optimized with the standard GRPO clipped policy-gradient objective with a KL penalty (full formulation in Appendix 6).

4 Experiment↩︎

5pt

4.1 Dataset↩︎

We evaluate on two benchmarks that test long-term conversational memory: LoCoMo [1] and LongMemEval-S [23]. LoCoMo contains multi-session dyadic conversations paired with QA annotations spanning four question categories: single-hop factual recall, temporal reasoning, open-domain, and multi-hop. Following the conversation-level split protocol of [29], we partition the ten conversations into a 1:1:8 train/validation/test split, yielding 152/81/1,307 QA pairs respectively (detailed statistics in Table 3). The training and validation sets are used exclusively for optimizing the query rewriter; all main results are reported on the held-out test set. LongMemEval-S [23] features exceptionally long interaction histories that require precise answer localization. We evaluate on five categories: Single-Session-User (S-U), Single-Session-Preference (S-P), Temporal-Reasoning (Tmp.), Knowledge-Update (K-U.), and Multi-Session (Mlt.).

4.2 Baselines↩︎

We compare PPRO against two families of baselines (detailed descriptions in Appendix 7.4). Training-free baselines include Full-Context, MemGPT [2], A-Mem [5], LightMem [45], Mem0 [4], and SimpleMem [6]; we adopt results reported by [6] under identical evaluation metrics. Training-based baselines include MEM1 [30] and MemAgent [31]; we deploy their official Qwen2.5-7B checkpoints and evaluate on our test split.

4.3 Evaluation Metrics↩︎

We use three metrics across the two benchmarks. Token-level F1 [46] measures the overlap between predicted and reference answer tokens; it is our primary metric on LoCoMo. BLEU-1 [44] serves as a secondary metric that rewards precision and penalizes irrelevant tokens in the prediction. LLM-as-Judge Accuracy is used for LongMemEval-S: following [6], we employ gpt-4.1-mini to produce binary Correct/Wrong labels and report accuracy.

4.4 Experiment Setting↩︎

To ensure fair comparison, we control the backbone of the answer model in PPRO to be identical to that of the baselines in each evaluation group. For LoCoMo, we evaluate with GPT-4o, Qwen2.5-3B, and Qwen3-8B as backbones for the comparison with training-free baselines, and Qwen2.5-7B for the comparison with training-based baselines. For LongMemEval-S, we use gpt-4.1-mini as the backbone following [6]. The rewriter model shares the same backbone as the answer model in each setting. Detailed hyperparameter settings, offline memory construction, and embedding models are provided in Appendix 7.3.

4.5 Main Results↩︎

Results on LoCoMo. Table 1 presents the main results on LoCoMo. In the comparison with training-free baselines, PPRO achieves the best overall F1 across all three backbones, consistently outperforming the previous best SimpleMem by 7–19 points. The gains are particularly pronounced on Temporal and MultiHop categories, confirming the effectiveness of our retrieval dimensions and profile-guided personalization. On GPT-4o, Full-Context achieves the highest SingleHop F1 as its strong long-context capability suffices for single-hop recall. On Qwen3-8B, SimpleMem outperforms PPRO on SingleHop due to its adaptive retrieval size that avoids context overload for smaller models.

Compared to training-based baselines, PPRO achieves the best overall F1, outperforming MemAgent by more than 5 points, with the largest gains in Temporal and SingleHop. MemAgent achieves a higher MultiHop F1, likely because it is trained on HotpotQA which directly reinforces multi-hop evidence chaining.

Results on LongMemEval-S. Table ¿tbl:tab:longmemeval95results? shows that PPRO achieves the best overall accuracy, outperforming all baselines. The advantage is most evident on Temporal-Reasoning and Multi-Session, demonstrating that our hierarchical memory generalizes to longer interaction histories. PPRO also achieves the best Single-Session-Preference accuracy, validating that the user profile effectively captures personalized preferences.

4.6 Ablation Study↩︎

We ablate each component by: (1) w/o Episodic Memory: removing episodic memory; (2) w/o Semantic Memory: removing semantic memory; (3) w/o Profile Injection: disabling profile-guided retrieval and removing profile context from answer generation; (4) w/o Retrieval Dimensions: disabling the retrieval dimensions in both rewriting and retrieval boosting; (5) w/o GRPO Training: removing GRPO training for the query rewriter.

As shown in Table [tbl:tab:ablation], all components contribute positively, with removing episodic memory causing the largest drop as it provides the primary evidence source. Removing semantic memory also leads to a substantial decline, confirming its complementary role in providing cross-dialogue aggregated context. Profile injection, retrieval dimensions, and GRPO training each provide consistent gains, demonstrating that they collectively enhance online retrieval quality through personalization, dimension matching, and learned query rewriting.

2pt

4.7 Discussion↩︎

Hyperparameter Sensitivity. We analyze the sensitivity of PPRO to two key hyperparameters.

Query-profile trade-off \(\lambda\). Figure 3 (a) shows how the balance between query and profile embeddings affects retrieval quality. The query remains the dominant retrieval signal as performance generally increases with \(\lambda\), yet injecting a small proportion of profile embedding (\(\lambda=0.8\)) achieves the best F1, surpassing query-only retrieval (\(\lambda=1\)).

Reward signal trade-off \(\alpha\). Figure 3 (b) shows how the balance between retrieval and answer rewards affects downstream performance. The optimal \(\alpha=0.2\) indicates that the retrieval reward should dominate the training signal, as it directly guides the rewriter toward better evidence coverage. The answer reward, while contributing a smaller proportion, still provides complementary guidance by ensuring the rewritten queries ultimately lead to better answer generation.

Retrieval Analysis. We further analyze retrieval quality from two perspectives.

Profile-Guided Retrieval Quality. We measure Evidence Recall—the fraction of ground-truth evidence turns covered by retrieved episodic memories at varying retrieval sizes (formal definition in Appendix 6.5). Figure 4 compares profile-guided retrieval against query-only retrieval on episodic memory. At small sizes, the two methods perform comparably, but as the retrieval size grows, profile-guided retrieval increasingly outperforms the baseline. This confirms that the profile embedding acts as a relevance prior that helps rank relevant memories higher when the candidate pool is large.

Memory Layer Interaction. Figure 5 shows the joint effect of episodic and semantic memory retrieval sizes. Both layers contribute positively with clear diminishing returns. Performance changes more rapidly when increasing the semantic retrieval size in the low-to-moderate range, indicating that semantic memory provides greater marginal benefit per retrieved item due to its higher information density from cross-dialogue aggregation. Moderate retrieval sizes achieve near-optimal performance without excessive context length.

4.8 Case Study↩︎

To qualitatively illustrate how each component contributes to the final answer, Table 2 presents a representative example where PPRO progressively resolves a temporally confounded query. As components are added, each addresses a distinct failure mode, and only the full system produces the correct answer. This demonstrates that profile-guided retrieval and retrieval-oriented rewriting are complementary—the profile resolves user-level ambiguity, while the trained rewriter resolves temporal and factual specificity.

5 Conclusion↩︎

We presented PPRO, a retrieval-centric framework that addresses user-agnostic ranking and task-agnostic optimization in memory-augmented conversational agents. PPRO constructs a hierarchical memory system and injects user profile embeddings as a personalized prior into retrieval scoring. A query rewriter trained with GRPO using evidence retrieval quality and answer quality as reward signals closes the optimization loop between query formulation and downstream performance without modifying memory banks or the answer model. Experiments on LoCoMo and LongMemEval-S demonstrate consistent improvements over both training-free and training-based baselines, confirming personalized retrieval optimization as a key factor in effective long-term memory use.

Limitations↩︎

Although PPRO achieves effective personalized retrieval through hierarchical memory and profile-guided optimization, several limitations remain.

Static Evaluation Setting. PPRO is evaluated on static benchmarks where all dialogues are given in advance. We do not evaluate in streaming scenarios where memories must be incrementally updated during ongoing interaction.

Supervised Reward Signal. GRPO training relies on ground-truth evidence annotations for the retrieval reward. Extending the optimization to unsupervised settings where such annotations are unavailable remains to be explored.

Single-User Scope. The current framework models each user independently. In multi-party or community settings, shared context and inter-user relationships could further inform retrieval but are not captured by our per-user profile design.

Future Work: We plan to extend PPRO to streaming incremental memory updates, explore self-supervised reward signals to remove annotation dependency, and investigate cross-user profile sharing for multi-party conversations.

Appendix↩︎

This appendix is organized as follows:

• Method details (Section 6)

• Experiment setup (Section 7)

• Detailed ablation study (Section 8)

• Additional case studies (Section 9)

• Prompts (Section 10)

• LLM usage statement (Section 11)

6 Method Details↩︎

6.1 Retrieval Reward↩︎

The retrieval reward \(R_{ret,i}\) measures how well the retrieved episodic memories cover the ground-truth evidence. Let \(\mathcal{V}_q\) denote the annotated evidence turn set for query \(q\), and let \(\hat{\mathcal{V}}_i = \bigcup_{e \in \mathcal{R}^E_i} \mathrm{src}(e)\) be the set of source turns from the retrieved episodic memories \(\mathcal{R}^E_i\). The retrieval reward is defined as: \[R_{ret,i} = F_1(\hat{\mathcal{V}}_i,\, \mathcal{V}_q) = \frac{2P_i \mathrm{Rec}_i}{P_i + \mathrm{Rec}_i},\] where \(P_i = |\hat{\mathcal{V}}_i \cap \mathcal{V}_q| / |\hat{\mathcal{V}}_i|\) and \(\mathrm{Rec}_i = |\hat{\mathcal{V}}_i \cap \mathcal{V}_q| / |\mathcal{V}_q|\).

6.2 EMA-based Reward Normalization↩︎

Because the retrieval reward \(R_{ret}\) and answer reward \(R_{ans}\) can have different scales and learning dynamics, we apply EMA-based adaptive normalization before combining them: \[\widetilde{R}_{*,i}=\frac{R_{*,i}}{\max(\mu_{*},\epsilon)},\] where \(\mu_{*}\) denotes the exponential moving average of each respective reward stream (\(* \in \{ret, ans\}\)). This normalization prevents one signal from dominating merely due to scale differences.

6.3 GRPO Objective↩︎

The GRPO objective optimizes the rewriter with a clipped policy-gradient surrogate and a KL penalty against a frozen reference policy \(\pi_{ref}\): \[\begin{align} \mathcal{L}_{GRPO}(\theta) = -\mathbb{E}_{i,t}\big[&\min\big(r_{i,t}\hat{A}_i,\; \mathrm{clip}(r_{i,t},1\!-\!\varepsilon, \\ 1\!+\!\varepsilon)\hat{A}_i\big) &- \mu D_{KL}\big], \end{align}\] where \(\mathbb{E}_{i,t}\) averages over all \(N\) sampled rewrites and their token positions, and the importance ratio is \[r_{i,t}=\frac{\pi_\theta(q'_{i,t}\mid q'_{i,<t},q,C)}{\pi_{\theta_{\mathrm{old}}}(q'_{i,t}\mid q'_{i,<t},q,C)}.\] The KL divergence is estimated token-wise following DeepSeekMath [21]. The answer model, embedding model, and memory banks remain frozen; gradients are applied only to the rewriter.

6.4 Retrieval Dimensions↩︎

Each episodic memory and each rewritten query is annotated with three retrieval dimensions:

• Tense: present, past, or future — indicates the temporal orientation of the statement.

• Subjectivity: objective or subjective — distinguishes factual statements from opinions or preferences.

• Certainty: certain or speculative — differentiates confirmed facts from plans or hypotheticals.

During episodic retrieval, each dimension of the rewritten query \(\delta_q\) is compared against the corresponding dimension of each candidate memory \(\delta_e\). A fixed bonus of \(0.05\) is added to the retrieval score for each matching dimension, yielding a maximum boost of \(0.15\) when all three dimensions align.

6.5 Evidence Recall↩︎

Evidence Recall measures how well the retrieved memories cover the ground-truth evidence for a given query. Each query \(q\) is annotated with a set of evidence turn identifiers \(\mathcal{V}_q\). Each retrieved memory \(e\) is associated with its source turn(s) \(\mathrm{src}(e)\) via provenance tracking. Given the retrieved episodic memories \(\mathcal{R}\) at a given retrieval size: \[\mathrm{EvidenceRecall} = \frac{|\mathcal{V}_q \cap \bigcup_{e \in \mathcal{R}} \mathrm{src}(e)|}{|\mathcal{V}_q|}\]

6.6 Algorithm↩︎

7 Experiment Setup↩︎

7.1 Dataset Split Statistics↩︎

LoCoMo. The LoCoMo dataset [1] contains 10 multi-session dyadic conversations, each spanning 19–35 sessions between two speakers. Questions are annotated into five categories: SingleHop (single-session factual recall), Temporal (temporal reasoning requiring date/order understanding), OpenDomain (requiring external or commonsense knowledge), MultiHop (cross-session evidence synthesis), and Adversarial (unanswerable questions). Following the conversation-level split protocol of [29], we assign conversations to train/validation/test splits at the conversation level (1:1:8 ratio) to prevent information leakage across splits. Category 5 (Adversarial) questions are excluded from all evaluations, as their gold answer is uniformly “unanswerable” and they do not test memory retrieval capability. The resulting statistics are shown in Table 3.

LongMemEval-S. LongMemEval [23] provides exceptionally long interaction histories (up to 500+ turns) and tests fine-grained answer localization. We use its short-form QA subset (LongMemEval-S) which contains 5 question categories: Single-Session-User (user-stated facts within one session), Single-Session-Preference (user preferences expressed in one session), Temporal-Reasoning (temporal ordering and date-based questions), Knowledge-Update (facts that evolve over time), and Multi-Session (evidence scattered across multiple sessions). We evaluate on this benchmark in a zero-shot manner without any fine-tuning on its data.

7.2 Evaluation Metrics↩︎

Token-level F1. Let \(\mathcal{T}_p\) and \(\mathcal{T}_r\) denote the multisets of whitespace-tokenized tokens in the prediction and reference respectively. Precision is \(|\mathcal{T}_p \cap \mathcal{T}_r|/|\mathcal{T}_p|\), recall is \(|\mathcal{T}_p \cap \mathcal{T}_r|/|\mathcal{T}_r|\), and F1 is their harmonic mean.

BLEU-1. Unigram BLEU with a brevity penalty, following [44]. It complements Token-level F1 by rewarding precision and penalizing irrelevant tokens in the prediction.

LLM-as-Judge Accuracy. For LongMemEval-S, gpt-4.1-mini evaluates each predicted answer against the ground-truth reference, producing a binary Correct/Wrong label based on semantic equivalence and temporal consistency. Accuracy is the percentage of Correct labels.

Implementation details. Tokenization and BLEU-1 computation use NLTK 3.8 [47] with word_tokenize and sentence_bleu (smoothing method 1). The BLEU-1 reward is computed with NLTK’s sentence_bleu (smoothing method 1). Sentence embeddings use bge-large-en [48] via the FlagEmbedding library. GRPO training is implemented with the VERL framework [49].

7.3 Hyperparameter Settings↩︎

Retrieval. We use bge-large-en as the embedding model. Episodic retrieval top-\(K\)=25, semantic retrieval top-\(K\)=7, similarity threshold 0.6, query-profile mixing weight \(\lambda\)=0.8, and dimension boost \(b_\delta\)=0.05.

Offline memory construction. All offline stages (episodic extraction, semantic aggregation, and profile generation) use DeepSeek-V3.2 [50] as the shared instruction-following model \(\mathcal{G}\). The semantic memory clustering threshold is \(\tau_c = 0.85\).

GRPO training. We use the VERL framework with 2\(\times\) NVIDIA A800 80GB GPUs. Learning rate \(1\times10^{-6}\), batch size 64, 30 epochs, group size \(N\)=8. Clip ratio \(\varepsilon\)=0.2, KL coefficient \(\mu\)=0.001, entropy coefficient 0.001, weight decay 0.01, gradient clipping 1.0. Answer reward weight \(\alpha\)=0.2, retrieval reward weight \((1-\alpha)\)=0.8. Loss aggregation uses token-mean mode. We validate before training and select the best checkpoint based on validation F1.

Computational resources. The rewriter model shares the same backbone as the answer model in each evaluation setting. The embedding model is bge-large-en (335M parameters). Offline memory construction uses qwen3-8b (8B parameters). GRPO training is conducted on 2\(\times\) NVIDIA A800 80GB GPUs; each training run takes approximately 15 hours (\(\sim\)​30 GPU hours in total).

7.4 Baseline Descriptions↩︎

Training-free baselines. Full-Context feeds the complete dialogue history to the answer model without explicit memory construction. MemGPT [2] manages memory through an operating-system-inspired virtual context mechanism. A-Mem [5] stores memories as structured notes connected through dynamic indexing and memory evolution. LightMem [45] separates lightweight online memory use from offline consolidation. Mem0 [4] dynamically extracts and consolidates salient information with graph-based memory representations. SimpleMem [6] applies semantic structured compression, online synthesis, and intent-aware retrieval planning.

Training-based baselines. MEM1 [30] learns to maintain a compact internal memory state via PPO-trained iterative think-search-answer loops. MemAgent [31] uses DAPO to train a recurrent memory compression agent that processes long contexts through sequential chunk-level updates. Both methods are trained on HotpotQA.

8 Detailed Ablation Study↩︎

Table 4 presents the per-category breakdown of the ablation study. The trends are consistent with the overall results in the main text. Removing episodic memory causes the largest degradation, especially on MultiHop (\(-12.23\) F1) where multiple fine-grained facts must be combined. Removing semantic memory impacts SingleHop and Temporal most, as aggregated timelines and counts directly serve these question types. Profile injection contributes most to OpenDomain (\(-4.85\) F1), where user-specific relational context is needed for disambiguation. Retrieval dimensions show the largest impact on Temporal (\(-4.88\) F1), confirming that tense matching effectively guides temporal retrieval. GRPO training benefits OpenDomain and Temporal most, as these categories require more specific query reformulation.

9 Additional Case Studies↩︎

Tables 5 and 6 present two additional case studies following the same progressive format as Table 2 in the main text. Each table shows how incrementally adding PPRO components resolves a distinct retrieval challenge.

Case 2: Distinguishing Motivation from Outcome (Table 5). The question asks why Audrey makes recycled jewelry, but the memory bank contains both the stated motivation (“creativity and sustainability”) and a downstream outcome (“donate profits to an animal shelter”). Without the full system, retrieval conflates these two aspects. Episodic retrieval alone surfaces the more salient outcome memory (donation), while semantic memory partially captures sustainability but merges it with other environmental actions. The GRPO-trained rewriter learns to reformulate the query to specifically target stated motivations rather than behavioral consequences, surfacing the exact “creativity and sustainability” memory.

Case 3: Stated Reason vs.Inferred Motivation (Table 6). James explicitly stated he “wanted to learn something new,” but his memory bank also contains cooking-related social events (cooking for friends) and self-improvement themes. The challenge is distinguishing what the user actually said from what could be inferred. Episodic retrieval without GRPO surfaces socially-oriented cooking memories; semantic memory picks up the self-improvement theme but drifts toward inference (“challenge himself”). The profile further reinforces self-improvement but introduces paraphrase drift. Only the GRPO-trained rewriter generates a query specific enough to retrieve the verbatim stated reason, demonstrating that retrieval-oriented optimization helps the system distinguish explicit statements from plausible inferences.

10 Prompts↩︎

Figures 7–[fig:prompt95rewrite] show the simplified prompts used in the offline memory construction stages and the online query rewriting stage.

11 sec:LLM32Usage32Statement↩︎

Large Language Models (LLMs) such as ChatGPT are used as general-purpose tools to improve readability and clarity of the manuscript, e.g., for grammar checking, LaTeX formatting, and restructuring sentences. No parts of the research idea, dataset design, or experimental results are generated or influenced by LLMs. All technical contributions and conclusions are solely those of the authors.

5pt

2pt

2pt

References↩︎

---

1. \(\dagger\) Corresponding author↩︎