- name
- memory-engine
- description
- Memory guardian for OpenClaw — three-layer anti-amnesia with capacity management. v6.0: MEMORY.md soft cap (5000 chars) + USER.md cap (1500 chars) prevent prompt explosion. Auto-snapshot before compaction/GC. Integrity monitoring restores from snapshot if MEMORY.md is wiped. Works alongside OpenClaw native memorySearch (vector+hybrid) when available, falls back to built-in FTS5 when not. Focuses on what native lacks: auto-write, session extraction, watcher daemon, GC, health checks, backup, capacity management. Layer 1 (system): cron every 1h + watcher detects resets <30s + integrity checks. Layer 2 (platform): memory-flush + session-memory hook. Layer 3 (agent): write/search/maintain/boot scripts. Use when: recalling past conversations, writing memory entries, checking memory health, maintaining search index. Triggers: 'remember', 'recall', 'what did we discuss', 'memory status', 'search memory', 'reindex'.
Memory Engine 🧠⚡
Persistent memory that survives session restarts — with three layers of protection against amnesia.
The Problem
AI agents wake up blank every session. Existing solutions either:
- Need embedding APIs (cost money, need keys)
- Only search (useless if nothing was written)
- Rely on the AI "remembering" to save (it won't)
Memory Engine solves all three.
v5.0 — Memory Guardian Architecture
v5.0 redefines Memory Engine's role: instead of a "full-stack memory system", it's now a memory guardian — handling everything OpenClaw native memory *doesn't* do.
| Feature | Native memorySearch | Memory Engine |
|---|---|---|
| Semantic vector search | ✅ (embedding API) | ❌ (deferred to native) |
| BM25 keyword search | ✅ (hybrid mode) | ✅ (FTS5 fallback) |
| Auto-write daily logs | ❌ | ✅ memory-write.js |
| Session reset detection | ❌ | ✅ memory-watcher.sh (<30s) |
| Active session extraction | ❌ | ✅ --active mode |
| Health check + GC | ❌ | ✅ memory-maintain.js |
| Session resume context | ❌ | ✅ memory-resume.js |
| GitHub backup/restore | ❌ | ✅ memory-backup.sh |
| Zero-dep writing | ❌ (needs embedding API) | ✅ (pure Node.js) |
Upgrade from v3/v4 (zero data loss)
node scripts/memory-migrate.js # preview changes
node scripts/memory-migrate.js --apply # apply (backs up config first)
node scripts/memory-migrate.js --rollback --apply # revert if neededThe migrate tool:
- Enables native
memorySearchwith hybrid search + temporal decay - Enables
session-memoryhook (saves session on /new and /reset) - Updates cron to 1h if still at 6h
- Preserves ALL existing data — your memory files, index, daily logs untouched
- FTS5 search remains as fallback if native isn't configured
No breaking changes: all v3/v4 commands still work. memory-search.js auto-detects native mode and adapts. memory-boot.js skips FTS5 indexing when native is active (saves time).
Three-Layer Anti-Amnesia Architecture
Layer 1: SYSTEM (runs without AI) Layer 2: PLATFORM (OpenClaw built-in)
┌─────────────────────────┐ ┌──────────────────────────┐
│ cron job (every 1h) │ │ memory-flush │
│ • rebuild search index │ │ • auto-triggers before │
│ • health check │ │ context compaction │
│ • auto-create daily log │ │ • forces AI to write │
│ if missing │ │ memory/YYYY-MM-DD.md │
│ • active session extract│ │ • user-invisible │
│ • ensure watcher alive │ │ │
│ │ │ ⚙️ Built into OpenClaw, │
│ 🔴 watcher daemon │ │ just needs config │
│ • polls every 30s │ └──────────────────────────┘
│ • detects session reset │
│ • extracts memory <30s │
│ • auto-started by cron │
│ │
│ ⚙️ Pure shell/node, │
│ no AI involvement │
└─────────────────────────┘
Layer 3: AGENT (AI calls these)
┌──────────────────────────────────────────────────┐
│ memory-write.js → append daily log / MEMORY.md│
│ memory-search.js → FTS5 search (native fallback)│
│ memory-index.js → build/update FTS5 index │
│ memory-maintain.js → stats / rebuild / prune │
│ memory-migrate.js → upgrade/rollback helper NEW│
│ │
│ 📋 Governed by AGENTS.md rules: │
│ Session start → health check + read MEMORY.md │
│ During conversation → log key events │
│ Session end → summarize + promote to MEMORY.md │
└──────────────────────────────────────────────────┘Why three layers? Any single layer can fail:
- AI forgets to write → cron + memory-flush still capture data
- Cron stops → AI + memory-flush still work
- Memory-flush doesn't trigger (short session) → AI + cron still work
All three failing simultaneously = near zero probability.
Setup (2 minutes)
Quick Setup (v5.0 — recommended for OpenClaw ≥ 2026.4)
# 1. Install cron (watcher + maintenance)
(crontab -l 2>/dev/null; echo "0 * * * * $(pwd)/scripts/memory-cron.sh") | crontab -
# 2. Auto-configure native search + hooks
node scripts/memory-migrate.js --apply
# 3. Restart gateway
openclaw gateway restartThat's it! Native memorySearch handles search, we handle everything else.
Classic Setup (FTS5 mode — for OpenClaw < 2026.4 or no embedding API)
# 1. Install FTS5 dependency
npm install -g better-sqlite3
# 2. Install cron job
(crontab -l 2>/dev/null; echo "0 * * * * $(pwd)/scripts/memory-cron.sh") | crontab -Step 3: Enable memory-flush (add to openclaw.json)
{
"agents": {
"defaults": {
"compaction": {
"memoryFlush": {
"enabled": true,
"softThresholdTokens": 4000,
"prompt": "Pre-compaction memory flush. Store durable memories in memory/YYYY-MM-DD.md (create memory/ if needed). APPEND only, do not overwrite. Include: key decisions, task outcomes, user preferences, project status changes. Format each entry as: - HH:MM [tag] description. If nothing worth storing, reply with NO_REPLY."
},
"reserveTokensFloor": 20000,
"postIndexSync": "async"
},
"heartbeat": {
"every": "30m"
}
}
}
}⚠️ Important: Do NOT instruct memory-flush to write MEMORY.md or run exec commands. During flush, OpenClaw restricts writes to memory/YYYY-MM-DD.md only — MEMORY.md, SOUL.md, AGENTS.md are read-only. Exec/shell commands may also be restricted. The cron job handles index rebuilding independently.Step 4: Add rules to AGENTS.md
## Session Startup (MANDATORY)
1. Run: node skills/memory-engine/scripts/memory-boot.js
(single command: health check + index update + MEMORY.md content)
2. Use memory-search.js for specific recall (NOT full file reads)
## During Conversation (MANDATORY)
- Key decisions: node skills/memory-engine/scripts/memory-write.js --today "decision" --tag decision
- Task completion: node skills/memory-engine/scripts/memory-write.js --today "done" --tag done
## Session End (MANDATORY)
1. node skills/memory-engine/scripts/memory-write.js --today "Session summary"
2. node skills/memory-engine/scripts/memory-write.js --core "durable fact" --section category
3. node skills/memory-engine/scripts/memory-index.jsCommands
Migrate (upgrade/rollback) — NEW in v5.0
node scripts/memory-migrate.js # preview: what would change
node scripts/memory-migrate.js --apply # apply changes (backs up config)
node scripts/memory-migrate.js --rollback --apply # revert to FTS5-only mode
node scripts/memory-migrate.js --status # show current mode (JSON)Enables native memorySearch, session-memory hook, optimal cron frequency. Config backup created automatically before changes. Zero data loss guaranteed.
Boot (session startup)
node scripts/memory-boot.js # health + index + MEMORY.md outputReplaces 3 separate calls (--status + memory-index.js + read MEMORY.md) with one command. Auto-recovers from corrupted database. v5.0: auto-detects native mode, skips FTS5 when native active.
Write (most important!)
# Daily log (auto-creates memory/YYYY-MM-DD.md)
node scripts/memory-write.js --today "Deployed v2.0 to production"
node scripts/memory-write.js --today "User prefers dark mode" --tag preference
# Long-term memory (appends to MEMORY.md, capacity-managed — 5000 char soft cap)
node scripts/memory-write.js --core "Stack: Next.js + PostgreSQL + Redis"
node scripts/memory-write.js --core "Deploy via Docker Compose" --section infrastructure
# User profile (appends to USER.md — 1500 char soft cap)
node scripts/memory-write.js --user "Prefers concise answers, dislikes verbose explanations"
# Snapshot MEMORY.md (before risky operations)
node scripts/memory-write.js --snapshot
# Health check (now includes capacity % for MEMORY.md and USER.md)
node scripts/memory-write.js --statusHealth output:
{
"hasTodayLog": true,
"hasCoreMemory": true,
"coreMemory": { "chars": 2651, "cap": 5000, "usage": "53%" },
"userProfile": { "chars": 380, "cap": 1500, "usage": "25%" },
"snapshots": 3,
"gapCount": 2,
"healthScore": 71,
"warnings": ["2 gaps in last 14 days"]
}Resume (zero-latency session recovery) — Enhanced in v4.0
node scripts/memory-resume.js # generate recovery summary (<2000 tokens)
node scripts/memory-resume.js --max-chars 1000 # shorter for tight budgetsUse in AGENTS.md as the first session startup step. Outputs:
- Last session's final topic & status
- Recent 3 days' key entries (priority-sorted by tag)
- Active MEMORY.md sections summary
- NEW v4.0: Auto-detects unextracted reset sessions and extracts them before resuming
(eliminates the "resume misses recent work" gap)
Search (0 tokens, ~300 token results)
node scripts/memory-search.js "deployment plan" # top 3, 200 chars
node scripts/memory-search.js "API重构" --json --max 5 # CJK support
node scripts/memory-search.js "query" --max 1 --max-chars 100 # ultra-minimal
# Date filtering (auto-detects "4月2日" / "April 2" in queries)
node scripts/memory-search.js "4月2日完成了什么" # auto date filter
node scripts/memory-search.js "tasks" --date 2026-04-02 # exact date
node scripts/memory-search.js "progress" --recent 7 # last 7 days
node scripts/memory-search.js "bugs" --after 2026-04-01 --before 2026-04-08
# Last N entries (no query needed, time-ordered) — NEW in v3.0
node scripts/memory-search.js --last 5 # last 5 entries across all logs
node scripts/memory-search.js --last 10 --today # today's last 10
node scripts/memory-search.js --last 3 --tag done # last 3 [done] entries
node scripts/memory-search.js --last 5 --tag decision # last 5 decisionsIndex
node scripts/memory-index.js # incremental (skips unchanged)
node scripts/memory-index.js --force # full rebuildMaintain
node scripts/memory-maintain.js # show stats
node scripts/memory-maintain.js --reindex # force rebuild
node scripts/memory-maintain.js --prune-days 90 # trim old entries
# GC: detect & remove stale MEMORY.md entries — NEW in v3.0
node scripts/memory-maintain.js --gc # preview stale entries
node scripts/memory-maintain.js --gc --apply # remove themGC detects: old version references (>14d), completed TODOs, entries >60d old, duplicates.
Consolidate (capacity management) — NEW in v6.0
node scripts/memory-maintain.js --consolidate # analyze merge candidates
node scripts/memory-maintain.js --consolidate --apply # remove duplicates (snapshots first)When MEMORY.md exceeds 70% of the 5000-char soft cap, use this to find and merge duplicate entries. Auto-snapshots MEMORY.md before any destructive operation. For deeper consolidation (merging semantically similar entries), manually edit MEMORY.md.
Auto-Extract (session transcript mining) — Enhanced in v4.0
node scripts/memory-auto-extract.js # extract from active session (full scan)
node scripts/memory-auto-extract.js <path.jsonl> # extract from specific file
node scripts/memory-auto-extract.js --scan # scan all unprocessed reset sessions
node scripts/memory-auto-extract.js --active # incremental extract from active sessions (NEW v4.0)
node scripts/memory-auto-extract.js --dry-run # preview without writingExtracts: git commits, file writes, deployments, PM2 restarts, user instructions, test results, API responses, config changes, version releases, bug fixes (NEW v4.0). --active mode reads only new bytes since last extraction (offset-based), safe for frequent runs. Integrated into cron: auto-scans reset sessions + active sessions every 1h.
Watcher (real-time session reset detection) — NEW in v4.0
bash scripts/memory-watcher.sh # start daemon (polls every 30s)
bash scripts/memory-watcher.sh --once # single check, no loop- Detects new
.reset.files within 30 seconds - Immediately extracts memory + rebuilds index
- Auto-started by cron (ensures watcher is always running)
- PID managed via
.memory/watcher.pid - Solves the #1 failure mode: session reset → instant amnesia
Compact (compress old logs)
node scripts/memory-compact.js --stats # show candidates
node scripts/memory-compact.js --older-than 30 --dry-run # preview
node scripts/memory-compact.js --older-than 30 # execute (originals → memory/archive/)Compaction extracts headings + key bullets (✅/🔴/重要), saves originals to memory/archive/. Typical savings: 60-70%.
How Search Works
- Chunking: ~300 chars per chunk, 60 char overlap, split at headings
- Indexing: SQLite FTS5 full-text index + file metadata
- Search: BM25 + phrase matching for English; TF-density bigram for CJK
- Date filtering: Auto-detects dates in queries (4月2日, April 2, 2026-04-02)
- Ranking: TF_density × coverage × core_boost(1.5×) × temporal_decay(30-day half-life)
- Output: truncated snippets with
file#linecitations
Token Budget
| Operation | Tokens |
|---|---|
| Search (SQLite) | 0 |
| Results (3 × 200 chars) | ~300 |
| Read full daily file | ~2000 |
| Read 4 daily files | ~8000 |
| Savings | ~95% |
Backup & Disaster Recovery
Auto-backup (recommended)
Memory-cron automatically pushes to GitHub every 6 hours (if git remote is configured):
# One-time setup:
cd ~/.openclaw/workspace
git init && git branch -M main
git remote add origin https://<token>@github.com/<user>/openclaw-workspace.git
bash skills/memory-engine/scripts/memory-backup.sh # first push
# After this, cron handles it automatically every 6hRestore after reinstall
# After fresh OpenClaw install:
bash memory-restore.sh https://<token>@github.com/<user>/openclaw-workspace.git
# Restores: memories, config, skills, crontab, rebuilds search indexWhat's backed up
MEMORY.md+memory/*.md(all memories — irreplaceable)SOUL.md,USER.md,IDENTITY.md(personality)AGENTS.md,TOOLS.md,HEARTBEAT.md(behavior rules)skills/(all installed skills)openclaw.json→ backup copycrontab→ backup copy
What's NOT backed up (rebuilt automatically)
.memory/index.sqlite(rebuilt from md files by restore script).memory/cron.log(transient)
Requirements
v5.0 Native mode (recommended)
- Node.js 18+
- OpenClaw ≥ 2026.4 (for native memorySearch + session-memory hook)
better-sqlite3optional (only needed for FTS5 fallback)- No API keys required (OpenClaw auto-detects embedding providers)
Classic FTS5 mode
- Node.js 18+
better-sqlite3(for search/index)- Write + health check: zero dependencies
- No API keys, no cloud, no Docker