TencentDB Agent Memory Is a Permission System Wearing a Memory Costume

SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | October 14, 2026

The promise: by the end of this issue you will know exactly how TencentDB Agent Memory turns chat logs, docs, and code into governed assets, where its real engineering lives, and the two places it will hurt you in production.

The fence: we cover the v2.0.0 team memory stack (MemoryCore, MemoryProxy, MemoryKnowledge, Memory Hub), its L0 to L3 memory pyramid, its access model, and how it compares to MemGPT and MemoryBank. We skip the v1 single-agent OpenClaw plugin internals, Tencent Cloud's managed offering, and pricing.

What It Actually Does

Strip the marketing and TencentDB Agent Memory is three services and a control panel:

MemoryCore (port 8420): a Node.js gateway over SQLite that stores conversations, distills them into layered memory, versions Skills, and holds every ownership and ACL record. Its own API doc counts 108 endpoints, all POST, RPC style.
MemoryProxy (port 8096): an Anthropic and OpenAI compatible reverse proxy. You point Claude Code, Codex, CodeBuddy, OpenClaw, or Hermes at it, and it injects memory into the system prompt before forwarding to the real model.
MemoryKnowledge (port 8424): builds Wiki pages from documents and a CodeGraph (symbols, callers, impact paths) from repositories.
Memory Hub (port 8125): the panel where humans create Teams and Agents, review assets, and decide who gets what.

The output is four asset types: Chat Memory, Skill, LLM Wiki, CodeGraph. Each carries an owner, version, status, usage count, and a visibility of private, team, restricted, or agent.

Community signal is loud: 28.6k stars, 2.8k forks, 260 open issues, 522 open PRs, MIT licensed, Node 22.16 or newer. The only benchmark in the v2 README: PersonaMem rises from 48% to 76% with memory enabled.

The one line summary: RAG answers "can we find it." This system answers "who may use it, which version is valid, and which agent should receive it." That second question is the product.

The Architecture, Unpacked

Caption: Focus on the proxy. Memory is not a library your agent calls, it is a hop every token passes through, and the permission check happens before retrieval, not after.

Three decisions carry the design.

Layered distillation over flat storage. Raw turns land in L0, then an async pipeline extracts L1 atoms (facts, preferences, constraints), aggregates them into L2 scenario files, and periodically rewrites an L3 persona. Retrieval inverts the order: L2 and L3 bootstrap context cheaply, and only when a precise fact is needed does hybrid search (BM25 plus vectors, fused with Reciprocal Rank Fusion, RRF, which merges ranked lists by summing 1/(k + rank)) drop to L1 or L0. The upper layers are compressed, but the path back to ground truth is never deleted.

Scope first, search second. Every data plane call carries a team_id + agent_id + user_id triple. Memory Hub's Fixed Binding plus ACL narrows the candidate set by team, user, agent, and visibility before the query runs. This is the opposite of most RAG stacks, which retrieve broadly and filter after.

Knowledge as tools, not context. Wiki and CodeGraph are not stuffed into prompts. Agents discover them via /v3/tools/list and pull pages, source, or impact paths via /v3/tools/call only when needed.

The Code, Annotated

Integration is an environment variable. This is the entire Claude Code setup from the project's install guide:

# No SDK, no plugin, no MCP server. Swap the base URL and the proxy owns the session.
export ANTHROPIC_BASE_URL=http://127.0.0.1:8096/claude-code/default  # ← THIS is the trick: memory lives in the network path
#                                                       ^^^^^^^ instance ID, becomes x-tdai-service-id downstream

# Use a BUSINESS user's key. The system_admin key cannot own Teams or Agents yet,
# so the session picker comes back empty. The install FAQ calls this out explicitly.
export ANTHROPIC_AUTH_TOKEN="sk-mem-<32 random chars>"

claude --model "$PROXY_UPSTREAM_MODEL"
# Proxy pipeline per request: auth → sessionInit (first turn only) → injection → upstream

Caption: Zero code integration works because the proxy speaks the agent's native protocol. The cost is that every prompt and completion now transits a service you operate.

What a custom adapter does. MemoryCore's README defines three responsibilities for any adapter: write turns to L0, recall before the next prompt, inject bounded context. Here they are against the documented v3 endpoints:

const CORE = "http://127.0.0.1:8420";
const headers = {
  "Content-Type": "application/json",
  "x-tdai-service-id": "default",        // memory instance (tenant boundary)
  "x-tdai-team-id": "t_platform",        // v3 ENFORCES team+agent+user; omit one and you
  "x-tdai-agent-id": "agt_reviewer",     // silently fall into a shared "default" bucket.
  "x-tdai-user-id": "u_mohinish",        // Pass all three on every call.
};

async function call(path: string, body: object) {
  const r = await fetch(`${CORE}${path}`, { method: "POST", headers, body: JSON.stringify(body) });
  const j = await r.json();
  if (j.code !== 0) throw new Error(`${path}: ${j.message}`); // envelope is HTTP 200 + code, check code
  return j.data;
}

// 1) Capture: write the finished turn to L0. Returns fast; L1 extraction runs async.
await call("/v3/conversation/add", {
  session_id: "sess_42",
  messages: [{ role: "user", content: "Don't refactor the old auth module, mobile still uses it." }],
}); // → { accepted_ids: ["msg_1"], accepted_versions: ["v1"], total_count: 1 }

// 2) Recall: precise facts come from L1, capped hard.
const atoms = await call("/v3/atomic/search", {
  query: "refactor auth module",
  type: "instruction",   // L1 atoms are typed: episodic | persona | instruction
  limit: 5,              // ← THIS is the trick: bounded recall. Memory must never flood the window.
});

// 3) Skills: ask the server to render the injection block under a character budget.
const skills = await call("/v3/skill/listing", {
  query: "code review",
  char_budget: 8000,     // default 8000 chars, roughly 2k tokens at ~4 chars/token
}); // → { mode: "search", listing: "<available_skills>...", hits: [...] }

Caption: Budgets are enforced server side (limit, char_budget), so a misbehaving adapter cannot blow up the prompt. The three ID headers are what make "team memory" not leak into "everyone's memory."

The knobs that decide your latency and cost. From the shipped tdai-gateway.yaml:

memory:
  pipeline:
    everyNConversations: 5        # L1 extraction is batched, not per turn: fewer LLM calls
    l1IdleTimeoutSeconds: 600     # ...or flushed after 10 min idle so short sessions still distill
    l2DelayAfterL1Seconds: 90     # scenes wait for atoms to settle
    l2MinIntervalSeconds: 900     # L2 rewrites throttled to between 15 and 60 min
    l2MaxIntervalSeconds: 3600
  persona:
    triggerEveryN: 50             # L3 persona rewritten rarely: it is the most expensive, least volatile layer
    maxScenes: 15
  recall:
    maxResults: 5
    scoreThreshold: 0.3           # below this, inject nothing; silence beats noise
    strategy: "hybrid"
    timeoutMs: 5000               # ← THIS is your hot path budget. Recall sits in front of every turn.
  embedding:
    provider: "none"              # vectors OFF by default: "hybrid" quietly degrades to BM25 only
  bm25:
    language: "zh"                # default tokenizer targets Chinese. English teams: change this first.

Caption: Every LLM call in the pipeline is amortized (batched L1, throttled L2, rare L3). The one thing that is not amortized is recall, and it ships with a five second ceiling.

It In Action

Input: a two agent team, "Builder" and "Reviewer," inside Team t_platform. On Monday, the human tells Builder in Claude Code:

❝

"Don't refactor the old auth module, mobile is still using it."

Step one, capture. The proxy authenticates the sk-mem key, finds the session already bound to Team t_platform, Agent Builder, and forwards the turn. On the way back it writes both messages to L0 via /v3/conversation/add. One row each, version: v1.

Step two, distill. Per defaults, L1 extraction fires after five captured conversations or 600 s idle. The extractor LLM (default config ships deepseek-v3.2, maxTokens: 32000) emits an atom:

{ "id": "rec_17", "version": 1, "type": "instruction",
  "content": "Do not refactor legacy auth module; mobile client depends on it.",
  "team_id": "t_platform", "agent_id": "agt_builder" }

Ninety seconds later, L2 folds it into a scenario file such as projects/auth-service.md. L3 persona will not change until 50 conversations accumulate.

Step three, share. The atom lives in Builder's Chat Memory, which is private by default. In Memory Hub, the human flips that Chat Memory asset to team and binds it to Reviewer. Nothing is shared until a human makes that click.

Step four, recall in a different agent. Wednesday, a new Claude Code session as Reviewer: "Clean up auth, it's a mess." The proxy's injection hook pulls Reviewer's bound L2/L3 context, runs a scoped recall (max 5 results, threshold 0.3), and asks /v3/skill/listing for matching Skills. The upstream model receives a system prompt that already contains the constraint.

Output: Reviewer replies that the legacy auth module is off limits because mobile depends on it, and proposes wrapping it instead. No human repeated anything.

The numbers that bound this flow:

Stage	Default bound	Where it bites
Recall results injected	5 items, score ≥ 0.3	Below threshold, nothing is injected
Recall latency ceiling	5,000 ms	Added to time to first token on a cold path
Skill block	8,000 chars (about 2k tokens, estimate)	Fixed tax on every turn if skills match
L1 freshness	up to 5 conversations or 10 min	A constraint said once may not be recallable in the same hour
Proxy session cache TTL	1,800 s (Redis)	Rebinding after 30 min idle

Caption: The constraint crossed agents in about ten minutes of wall clock and one human click. The click is the feature, and the ten minutes is the tax.

Why This Design Works, and What It Trades Away

Why it works. MemGPT proved that tiered memory beats stuffing: on Deep Memory Retrieval, GPT-4 jumped from 32.1% to 92.5% accuracy and GPT-4 Turbo from 35.3% to 93.4% once the model could page history in on demand. TencentDB keeps the tiers but moves the paging decision out of the model and into infrastructure. That makes behavior predictable across models with weak tool calling, a real problem: MemGPT itself degraded sharply on GPT-3.5 because of poor function calling.

The scope first ACL model is also correct for teams. Filtering after retrieval leaks through ranking side effects and snippets. Filtering before retrieval cannot leak what was never a candidate.

What it trades away.

Latency on the hot path. Recall plus Skill rendering sits between the agent and the model on every turn, with a 5 s ceiling.
Freshness. Batched extraction means memory lags conversation. That is the price of not running an LLM per turn.
Autonomy. The agent cannot decide "I should remember this" mid task the way MemGPT's core_memory_append can. Distillation is a background job, not a reasoning step.
Trust surface. Every token transits the proxy. The proxy's own v3 doc states that session/refresh-cache, session/force-archive-skill, and the rate limit admin API run without authentication, and proxy-destroy is public if admin.apiKey is unset. Bind to loopback or put it behind your own gateway.

Technical Moats

The algorithms are not the moat. Layered summaries trace back to MemoryBank and MemGPT, CodeGraph reuses code from colbymchenry/codegraph, Skill management reuses parts of Hermes Agent, and Wiki follows Karpathy's LLM Wiki pattern. The team credits all three openly.

The moat is the data model. 108 MemoryCore endpoints encode a governance schema: four isolation IDs, four visibility levels, six ACL permissions (read, write, delete, assign, share, use), allow and deny effects, per agent asset bindings with priority and injection mode, optimistic locking on Skills, and generation logs that trace every L1 to L3 artifact back to its source. Replicating "a vector store with summaries" takes a weekend. Replicating an audited ownership model that a security team will sign off on takes quarters.

The second moat is distribution. A drop in base URL means the integration cost for any Anthropic or OpenAI compatible agent is zero, and 28.6k stars plus 522 open PRs say the community is already writing adapters.

Contrarian Insights

Insight One: the benchmark everyone quotes does not measure the product. Coverage of this repo leads with "61% fewer tokens" (WideSearch tokens down from 221M to 85M, success up from 33% to 50%). Those numbers come from the v1 symbolic short term memory: Mermaid task canvases and context offloading inside a single agent. The v2 README, which pitches team sharing, publishes exactly one benchmark: PersonaMem, a single user personalization test. There is no published number for the thing that is new, cross agent transfer. Nobody has measured whether Reviewer actually gets better because Builder learned something, or whether the injected memory is net positive after its token tax.

Insight Two: a memory hub that never forgets is a liability with a timestamp. MemoryBank (Zhong et al.) modeled forgetting on the Ebbinghaus curve, decaying retention with elapsed time and reinforcing memories each time they are recalled. TencentDB has no decay. Old Skill versions default to versionTtlDays: 0, never expire, and atoms persist until a human deletes them. "Mobile still uses the old auth module" is true in October and dangerous in March. The system's answer is human review in the panel, which works for ten agents and collapses at a hundred. Governance without decay just means you can audit exactly which stale fact broke production.

Takeaway

TencentDB Agent Memory inverts MemGPT. MemGPT's thesis was that the LLM should manage its own memory through function calls, paging data in at a 70% context warning and flushing at 100%. TencentDB takes memory management away from the model entirely: a proxy decides what enters the prompt, a background pipeline decides what gets distilled, and a human decides what gets shared. The research arc went from "let the model be the operating system" to "the model is just a process, and the kernel lives in the network path."

TL;DR For Engineers

Integration is one env var: ANTHROPIC_BASE_URL pointed at MemoryProxy on 8096. Every token now transits it, so treat it as production infrastructure.
Memory is a four level pyramid (L0 raw, L1 atoms, L2 scenes, L3 persona). Upper layers bootstrap, lower layers verify, nothing is irreversibly compressed.
Access is scoped before retrieval by team_id + agent_id + user_id, and new memories are private until a human shares them.
Defaults to change on day one: enable embeddings (vectors are off), switch bm25.language from zh, set admin.apiKey, use a business user key, not admin.
The 61% token savings headline is from v1 single agent memory. v2's team sharing is unbenchmarked.

Explain It Like I'm New

Imagine hiring a brilliant contractor who forgets everything the moment they leave the building. Every morning you re explain the project, the quirks, the things not to touch. Now imagine you have five of these contractors, and none of them talk to each other.

That is what working with AI agents is like today. Each session starts from zero, and whatever one agent figured out stays trapped in its chat history.

TencentDB Agent Memory is a shared filing system for an AI team. It quietly sits between your agents and the AI model, saves what gets said, and over time boils long conversations down into short, useful notes: preferences, decisions, how to do recurring tasks. It also turns your documents and code into something agents can look up instead of rereading.

The interesting part is not the remembering. It is the permissions. Every note has an owner, a version, and a list of who may use it. Notes start private, and a human decides when one becomes team knowledge, much like a company deciding which internal docs go on the shared drive.

This matters because the next phase of AI at work is not one smart assistant. It is many agents working together, and they will only be useful if what one learns can safely reach the others.

See It In Action

Memory Hub demo video, TencentCloud (GitHub README). github.com/TencentCloud/TencentDB-Agent-Memory. The official walkthrough of creating a team, equipping agents with assets, and watching Chat Memory grow in the panel. The fastest way to see the governance model rather than read about it.
Introduction to MemGPT, Charles Packer at the Zilliz Unstructured Data Meetup. Talk writeup with YouTube link. The MemGPT author explains virtual context management and the OS analogy that TencentDB's tiered design builds on.
How Letta is advancing AI with long term memory, AI Agent Podcast (Composio) with Charles Packer. composio.dev. Covers how MemGPT evolved into Letta and the production lessons of self managed memory, a useful counterpoint to TencentDB's proxy managed approach.
MemGPT: Towards LLMs as Operating Systems, alphaXiv audio lecture. alphaxiv.org/audio/2310.08560. A compact spoken walkthrough of main context, external context, and the paging control flow.
Hands on with Tencent Cloud Agent Memory, dev.to (liuyuyan6100). dev.to. A practitioner run of the local stack covering how L0 to L3 grows and how retrieval and governance interact.

Community Conversation

Andrej Karpathy, LLM Knowledge Bases post on X and LLM Wiki gist. The idea file behind TencentDB's Wiki asset: compile documents into LLM maintained markdown pages and query those instead of re retrieving raw chunks.
VentureBeat, coverage of the Karpathy pattern. Captures the practitioner debate, including founders arguing the jump from a personal wiki to enterprise operations is where it gets hard, exactly the gap TencentDB's ACL layer targets.
Bryan Vargas, LinkedIn commentary. Frames the repo as a reference architecture for asset separation and flags the right first test: verify which model the extraction step calls, since memory should never silently leave your machine.
Regolo.ai, complete guide for Hermes and OpenClaw. The source most people cite for the WideSearch numbers, and it positions TencentDB against Mem0 and OpenViking.
Agentic AI Foundation, Karpathy's LLM Wiki as Agent Memory. Maps the wiki pattern onto semantic, procedural, and summary memory, and stresses that any wiki needs maintenance passes, the missing "forgetting" in Insight Two.
QbitAI (量子位), launch coverage. Reports Tencent's own scenario numbers: up to 61% fewer tokens in web search, 33% in code repair, 31% in long document tasks, from the v1 short term memory work.

— # (#)

Memory Is Becoming Infrastructure, and Infrastructure Needs Owners

TencentDB Agent Memory is not the smartest memory system you can build. MemGPT's self directed paging is more elegant, and MemoryBank's forgetting curve is more honest about time. What Tencent shipped is the first open source memory layer that treats agent knowledge the way a platform team treats secrets or config: owned, versioned, scoped, and revocable. That is the right bet. The next production incident in a multi agent system will not be an agent forgetting something. It will be an agent remembering something it should never have been allowed to see, or something that stopped being true three months ago. This stack solves the first problem well and has barely started on the second.

References

MemGPT: Towards LLMs as Operating Systems, Packer et al., UC Berkeley. Virtual context management; DMR accuracy 32.1% to 92.5% on GPT-4.
MemoryBank: Enhancing Large Language Models with Long Term Memory, Zhong et al. Ebbinghaus inspired forgetting and reinforcement for LLM memory.
Lost in the Middle: How Language Models Use Long Contexts, Liu et al. Why bigger context windows do not replace retrieval.
Generative Agents: Interactive Simulacra of Human Behavior, Park et al. Memory stream, reflection, and retrieval for agents.
Beyond Goldfish Memory: Long Term Open Domain Conversation, Xu et al. The Multi Session Chat dataset behind MemGPT's evaluation.
Primary sources: TencentDB Agent Memory repo, INSTALL.md, MemoryCore v3 API, MemoryProxy v3 API, project site.

TencentDB Agent Memory turns agent conversations, docs, and code into four governed assets behind a drop in LLM proxy, layering memory from raw turns up to a persona and scoping every lookup by team, agent, and user before retrieval. Its real innovation is ownership and access control, not recall, and the open questions are cross agent benchmarks and forgetting.

Sponsored Ad If you enjoy practical AI insights, check out SnackOnAI and support the newsletter by subscribing, sharing, and exploring our sponsored ad—it helps us keep building and delivering value 🚀

Leave Granola and get up to 12 months free of Wispr Flow Notetaker + Dictation

If you have paid time left on an individual Granola plan, we'll match it with a Wispr Flow subscription that includes Notetaker and dictation, and add bonus time, up to 12 months total. Sign in or create a Wispr account and submit proof of your plan to check eligibility.

Try Notetaker Free