self-evolving-memory

name: self-evolving-memory description: > Self-evolving memory architecture for AI agents that learn across interactions without training. Use when building agents that need to: (1) accumulate experience across cases, (2) correct recurrent reasoning mistakes via reflection, (3) adapt tool-use behavior dynamically. Applicable to medical diagnosis, code debugging, customer support, research analysis, and any domain where agents solve repeated similar tasks. Trigger: self-evolving memory, inter-case learning, agent memory, episodic memory, adaptive heuristics, tool reliability tracking, agent reflection.

Self-Evolving Memory Architecture

Pattern from Evo-MedAgent (arXiv:2604.14475) — a test-time memory module that equips any frozen LLM agent with inter-case learning capacity, requiring no training.

Core Architecture

Three complementary memory stores:

1. Retrospective Clinical Episodes (RCE)

• Store problem-solving experiences from past cases as structured episodes
• Each episode: {problem_description, solution_path, outcome, key_insights}
• At test time: retrieve top-k similar episodes for the current case
• Similarity: semantic embedding matching on problem description

2. Adaptive Procedural Heuristics (APH)

• Priority-tagged diagnostic/decision rules that evolve via reflection
• Format: {rule, priority, confidence, last_updated, source_case}
• After each case: reflect on what worked/didn't, update heuristics
• Rules are sorted by priority; higher confidence = higher priority

3. Tool Reliability Controller (TRC)

• Track per-tool trustworthiness scores over time
• Format: {tool_name, success_rate, avg_quality, last_used}
• Dynamically adjust tool selection based on reliability scores
• Decay old scores to adapt to tool updates/changes

Workflow

Step 1: Case Processing

Input: new problem/case
1. Query RCE for similar past episodes (embedding similarity)
2. Query APH for relevant heuristics (keyword/category match)
3. Query TRC for tool reliability scores
4. Combine retrieved context into augmented prompt
5. Execute agent reasoning with augmented context

Step 2: Post-Case Reflection

After case completion:
1. Evaluate outcome (correct/incorrect/partial)
2. If incorrect: analyze failure mode, create new heuristic
3. If correct: reinforce existing heuristics, store episode in RCE
4. Update TRC scores for tools used
5. Prune low-confidence heuristics periodically

Step 3: Memory Maintenance

Periodically (every N cases):
1. Cluster similar RCE episodes, merge duplicates
2. Recalculate APH priorities based on aggregate outcomes
3. Decay TRC scores for unused tools
4. Remove heuristics with confidence < threshold

Implementation Notes

• No training required: works on top of any frozen model
• Per-case overhead: bounded by one retrieval pass + one reflection call
• Scalability: use vector database for RCE retrieval at scale
• Safety: reflection should validate new heuristics before adding

Performance Gains (from paper)

Related Papers

• DeepMedix-R1 (arXiv:2509.03906): RL with grounded rewards for explainable medical reasoning
• Rational QM (arXiv:2510.02877): Finite qubit capacity limits in discretized Hilbert space