Florian Hartmann
b83546d833
Architecture (Agent 1):
- hermes_memory/tier2/{schema,facts,entities,relations,timeline}.py
- hermes_memory/tier3/{backend,chroma_backend,embedder}.py
- hermes_memory/graph/nx_store.py
- hermes_memory/api/memory_api.py (unified API)
- hermes_memory/cron/{consolidate,embed_queue,graph_refresh,prune}.py
- hermes_memory/config.py + pyproject.toml
Integration Plan (Agent 3):
- INTEGRATION_PLAN.md: Memory Provider Plugin strategy
- Hermes Core needs minimal changes
- sync_turn() + prefetch() hooks
- Skills integration via nextlevel_search/remember
Auto-Extraction (Agent 2):
- ARCHITECTURE.md: Full extraction pipeline docs
- Chunking, Pre-Filter, LLM Prompts, Classification
- Entity-Linking, Temporal Reasoning, Deduplication
All files: Python syntax checked, ECC standards applied.
883 lines
30 KiB
Markdown
883 lines
30 KiB
Markdown
# Hermes Memory Next Level — Technische Architektur
|
|
|
|
**Version:** 1.0.0 │ **Autor:** Architektur-Experte │ **Datum:** 2026-06-03
|
|
|
|
---
|
|
|
|
## 1. Executive Summary
|
|
|
|
Hermes Memory Next Level (HMNL) ist ein mehrschichtiges, lokal laufendes Memory-Upgrade für Hermes Agent. Es erweitert das bestehende Key-Value Memory (Tier 1) um eine relationale Wissensbasis (Tier 2, SQLite) und eine semantische Vektorsuche (Tier 3, Qdrant/Chroma) mit Graph-Reasoning (NetworkX). Alle Tiers sind optional aktivierbar, lokal betreibbar und cloud-unabhängig.
|
|
|
|
---
|
|
|
|
## 2. Design-Prinzipien (ECC-Standard)
|
|
|
|
| Prinzip │ Beschreibung
|
|
| Einfachheit │ Jedes Tier kann standalone betrieben werden
|
|
| Kompaktheit │ SQLite-Tabellen mit │-Trennern, minimale Spaltenzahl
|
|
| Erweiterbarkeit │ Plugin-Architektur für neue Memory-Provider
|
|
| Lokalisierung │ Keine Cloud-Abhängigkeit, alles on-premise
|
|
| Konsistenz │ Einheitliche API über alle Tiers hinweg
|
|
|
|
---
|
|
|
|
## 3. Tier-Architektur
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ TIER 1 — Curated Memory (Bestehend) │
|
|
│ MEMORY.md │ USER.md │ §-delimited │ Frozen Snapshot │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ TIER 2 — Structured Knowledge (Neu) │
|
|
│ SQLite │ Fakten │ Entitäten │ Relationen │ Zeitachse │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ TIER 3 — Semantic Memory (Neu) │
|
|
│ Qdrant/Chroma │ Embeddings │ Ähnlichkeitssuche │ Cluster │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ GRAPH — Knowledge Graph (Neu) │
|
|
│ NetworkX │ Entitäten als Nodes │ Relationen als Edges │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ API — Unified Memory Interface │
|
|
│ Python-API │ Tool-Integration │ Cronjob │ Skills │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Modul-Struktur
|
|
|
|
```
|
|
hermes_memory/
|
|
│
|
|
├── __init__.py # Public API exports
|
|
├── config.py # Konfiguration & Defaults
|
|
│
|
|
├── tier1/ # Curated Memory (Wrapper)
|
|
│ ├── __init__.py
|
|
│ ├── curated_store.py # MEMORY.md / USER.md Interface
|
|
│ └── snapshot.py # Frozen Snapshot Management
|
|
│
|
|
├── tier2/ # Structured Knowledge (SQLite)
|
|
│ ├── __init__.py
|
|
│ ├── schema.py # DB-Schema & Migrationen
|
|
│ ├── connection.py # Pool & WAL-Handling
|
|
│ ├── facts.py # CRUD für Fakten
|
|
│ ├── entities.py # Entitäts-Verwaltung
|
|
│ ├── relations.py # Relationen-Management
|
|
│ ├── timeline.py # Zeitachsen-Queries
|
|
│ └── search.py # FTS5 & strukturierte Suche
|
|
│
|
|
├── tier3/ # Semantic Memory (Vektor-DB)
|
|
│ ├── __init__.py
|
|
│ ├── backend.py # Abstrakte Backend-Schnittstelle
|
|
│ ├── qdrant_backend.py # Qdrant-Implementierung
|
|
│ ├── chroma_backend.py # Chroma-Implementierung
|
|
│ ├── embedder.py # Embedding-Model Wrapper
|
|
│ ├── chunks.py # Text-Chunking-Strategien
|
|
│ └── semantic_search.py # Vektor-Suche & Reranking
|
|
│
|
|
├── graph/ # Knowledge Graph (NetworkX)
|
|
│ ├── __init__.py
|
|
│ ├── builder.py # Graph aus Tier 2 & 3 aufbauen
|
|
│ ├── nx_store.py # NetworkX Persistenz (GraphML)
|
|
│ ├── traversal.py # Pathfinding & Traversal
|
|
│ ├── centrality.py # Wichtige Knoten identifizieren
|
|
│ └── communities.py # Community Detection
|
|
│
|
|
├── api/ # Unified Interface
|
|
│ ├── __init__.py
|
|
│ ├── memory_api.py # Haupt-API-Klasse
|
|
│ ├── tool_adapter.py # Integration memory_tool.py
|
|
│ ├── session_adapter.py # Integration session_search_tool.py
|
|
│ ├── cron_adapter.py # Integration cron/scheduler.py
|
|
│ └── skill_adapter.py # Integration skills_system
|
|
│
|
|
├── cron/ # Hintergrund-Jobs
|
|
│ ├── __init__.py
|
|
│ ├── consolidate.py # Fakten-Deduplizierung
|
|
│ ├── embed_queue.py # Embedding-Job-Queue
|
|
│ ├── graph_refresh.py # Graph-Rebuild
|
|
│ └── prune.py # Alte Daten ausdünnen
|
|
│
|
|
└── utils/
|
|
├── __init__.py
|
|
├── validators.py # Eingabe-Validierung
|
|
├── sanitizers.py # Content-Sanitization
|
|
└── hashing.py # Content-Hashing für Deduplizierung
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Datenbank-Schema (Tier 2 — SQLite)
|
|
|
|
### 5.1 Fakten-Tabelle
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS facts (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
uuid TEXT NOT NULL UNIQUE, -- Global eindeutige ID
|
|
content TEXT NOT NULL, -- Fakt als natürlicher Text
|
|
content_hash TEXT NOT NULL, -- SHA-256 für Deduplizierung
|
|
category TEXT, -- user │ project │ domain │ tool
|
|
confidence REAL DEFAULT 1.0, -- 0.0 .. 1.0
|
|
source_type TEXT NOT NULL, -- session │ memory │ tool │ cron │ user
|
|
source_id TEXT, -- session_id │ tool_name │ NULL
|
|
created_at REAL NOT NULL, -- Unix-Timestamp
|
|
updated_at REAL NOT NULL, -- Unix-Timestamp
|
|
expires_at REAL, -- TTL (optional)
|
|
access_count INTEGER DEFAULT 0, -- Nutzungshäufigkeit
|
|
last_accessed REAL, -- Letzter Zugriff
|
|
is_archived INTEGER DEFAULT 0 -- Soft-Delete
|
|
);
|
|
|
|
CREATE INDEX idx_facts_category ON facts(category);
|
|
CREATE INDEX idx_facts_source ON facts(source_type, source_id);
|
|
CREATE INDEX idx_facts_created ON facts(created_at DESC);
|
|
CREATE INDEX idx_facts_hash ON facts(content_hash);
|
|
CREATE INDEX idx_facts_confidence ON facts(confidence DESC);
|
|
```
|
|
|
|
### 5.2 Entitäten-Tabelle
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS entities (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
uuid TEXT NOT NULL UNIQUE,
|
|
name TEXT NOT NULL, -- Kanonischer Name
|
|
aliases TEXT, -- JSON-Array: ["Alias1", "Alias2"]
|
|
entity_type TEXT NOT NULL, -- person │ project │ tech │ org │ concept │ place
|
|
description TEXT, -- Kurzbeschreibung
|
|
first_seen REAL NOT NULL, -- Erstes Vorkommen
|
|
last_seen REAL NOT NULL, -- Letztes Vorkommen
|
|
occurrence_count INTEGER DEFAULT 1, -- Häufigkeit
|
|
metadata TEXT -- JSON: {"key": "value"}
|
|
);
|
|
|
|
CREATE INDEX idx_entities_name ON entities(name);
|
|
CREATE INDEX idx_entities_type ON entities(entity_type);
|
|
CREATE INDEX idx_entities_aliases ON entities(aliases); -- FTS5 für Aliase
|
|
```
|
|
|
|
### 5.3 Relationen-Tabelle
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS relations (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
uuid TEXT NOT NULL UNIQUE,
|
|
from_entity_id TEXT NOT NULL REFERENCES entities(uuid),
|
|
to_entity_id TEXT NOT NULL REFERENCES entities(uuid),
|
|
relation_type TEXT NOT NULL, -- works_on │ knows │ depends_on │ part_of │ related_to
|
|
strength REAL DEFAULT 1.0, -- 0.0 .. 1.0
|
|
evidence_fact_id TEXT REFERENCES facts(uuid), -- Begründender Fakt
|
|
created_at REAL NOT NULL,
|
|
updated_at REAL NOT NULL
|
|
);
|
|
|
|
CREATE INDEX idx_relations_from ON relations(from_entity_id);
|
|
CREATE INDEX idx_relations_to ON relations(to_entity_id);
|
|
CREATE INDEX idx_relations_type ON relations(relation_type);
|
|
```
|
|
|
|
### 5.4 Timeline / Ereignisse
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS timeline (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
uuid TEXT NOT NULL UNIQUE,
|
|
event_type TEXT NOT NULL, -- milestone │ decision │ error │ insight │ change
|
|
title TEXT NOT NULL,
|
|
description TEXT,
|
|
related_entities TEXT, -- JSON-Array von entity_uuids
|
|
related_facts TEXT, -- JSON-Array von fact_uuids
|
|
session_id TEXT, -- Herkunft
|
|
timestamp REAL NOT NULL,
|
|
importance REAL DEFAULT 0.5 -- 0.0 .. 1.0
|
|
);
|
|
|
|
CREATE INDEX idx_timeline_time ON timeline(timestamp DESC);
|
|
CREATE INDEX idx_timeline_type ON timeline(event_type);
|
|
```
|
|
|
|
### 5.5 FTS5 für Volltextsuche
|
|
|
|
```sql
|
|
CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts USING fts5(
|
|
content,
|
|
content_rowid='id',
|
|
tokenize='unicode61'
|
|
);
|
|
|
|
CREATE VIRTUAL TABLE IF NOT EXISTS entities_fts USING fts5(
|
|
name || ' ' || COALESCE(description, ''),
|
|
content_rowid='id',
|
|
tokenize='unicode61'
|
|
);
|
|
```
|
|
|
|
### 5.6 Schema-Versionierung
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS memory_schema_version (
|
|
version INTEGER NOT NULL,
|
|
applied_at REAL NOT NULL
|
|
);
|
|
```
|
|
|
|
---
|
|
|
|
## 6. Tier 3 — Vektor-DB Schema (Qdrant / Chroma)
|
|
|
|
### 6.1 Qdrant Collections
|
|
|
|
```python
|
|
# Collection: memory_chunks
|
|
{
|
|
"name": "memory_chunks",
|
|
"vectors": {
|
|
"size": 384, # all-MiniLM-L6-v2 oder local embedding
|
|
"distance": "Cosine"
|
|
},
|
|
"payload_schema": {
|
|
"chunk_id": {"type": "keyword"},
|
|
"fact_id": {"type": "keyword"}, # NULL wenn direkt aus Session
|
|
"session_id": {"type": "keyword"},
|
|
"message_id": {"type": "integer"}, # messages.id aus SQLite
|
|
"source_type": {"type": "keyword"}, # fact │ session │ memory │ tool
|
|
"category": {"type": "keyword"},
|
|
"timestamp": {"type": "float"},
|
|
"content_hash": {"type": "keyword"},
|
|
"text_preview": {"type": "text"} # Erste 200 Zeichen
|
|
}
|
|
}
|
|
|
|
# Collection: entity_embeddings
|
|
{
|
|
"name": "entity_embeddings",
|
|
"vectors": {
|
|
"size": 384,
|
|
"distance": "Cosine"
|
|
},
|
|
"payload_schema": {
|
|
"entity_id": {"type": "keyword"},
|
|
"entity_name": {"type": "keyword"},
|
|
"entity_type": {"type": "keyword"},
|
|
"description": {"type": "text"}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 6.2 Chroma Collections (Alternative)
|
|
|
|
```python
|
|
# Chroma-Äquivalent
|
|
client.create_collection(
|
|
name="memory_chunks",
|
|
metadata={"hnsw:space": "cosine"}
|
|
)
|
|
client.create_collection(
|
|
name="entity_embeddings",
|
|
metadata={"hnsw:space": "cosine"}
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## 7. Graph-Schema (NetworkX)
|
|
|
|
### 7.1 Node-Attribute
|
|
|
|
```python
|
|
{
|
|
"node_type": "entity", # entity │ fact │ session │ concept
|
|
"uuid": "ent-uuid",
|
|
"name": "Projekt Alpha",
|
|
"entity_type": "project", # nur bei entity-Nodes
|
|
"weight": 1.0, # Centrality / Wichtigkeit
|
|
"created_at": 1717420800.0,
|
|
"last_seen": 1717420800.0,
|
|
"metadata": {} # Zusätzliche Attribute
|
|
}
|
|
```
|
|
|
|
### 7.2 Edge-Attribute
|
|
|
|
```python
|
|
{
|
|
"relation_type": "depends_on", # aus relations-Tabelle
|
|
"strength": 0.85, # Gewicht
|
|
"evidence": "fact-uuid", # Begründung
|
|
"first_seen": 1717420800.0,
|
|
"last_seen": 1717420800.0,
|
|
"bidirectional": False
|
|
}
|
|
```
|
|
|
|
### 7.3 Persistenz
|
|
|
|
```python
|
|
# Speicherung als GraphML (XML-basiert, menschenlesbar)
|
|
nx.write_graphml(G, path / "knowledge_graph.graphml")
|
|
|
|
# Oder als Pickle für Performance
|
|
nx.write_gpickle(G, path / "knowledge_graph.gpickle")
|
|
```
|
|
|
|
---
|
|
|
|
## 8. API-Design (Unified Memory API)
|
|
|
|
### 8.1 Hauptklasse: MemoryAPI
|
|
|
|
```python
|
|
class MemoryAPI:
|
|
"""Unified interface for all memory tiers."""
|
|
|
|
def __init__(
|
|
self,
|
|
profile: str = "default",
|
|
tier2_enabled: bool = True,
|
|
tier3_enabled: bool = True,
|
|
graph_enabled: bool = True,
|
|
tier3_backend: str = "chroma", # "qdrant" | "chroma"
|
|
embedding_model: str = "local", # "local" | "openai" | "sentence-transformers"
|
|
):
|
|
...
|
|
|
|
# ── Tier 1: Curated ──
|
|
def curated_get(self, store: str = "memory") -> str: ...
|
|
def curated_add(self, content: str, store: str = "memory") -> dict: ...
|
|
def curated_replace(self, old: str, new: str, store: str = "memory") -> dict: ...
|
|
def curated_remove(self, substring: str, store: str = "memory") -> dict: ...
|
|
|
|
# ── Tier 2: Structured ──
|
|
def fact_store(self, content: str, category: str = "general",
|
|
confidence: float = 1.0, source: str = "user") -> dict: ...
|
|
def fact_query(self, query: str, category: str = None,
|
|
limit: int = 10, min_confidence: float = 0.5) -> list: ...
|
|
def fact_get(self, uuid: str) -> dict: ...
|
|
def fact_update(self, uuid: str, **fields) -> dict: ...
|
|
def fact_delete(self, uuid: str, soft: bool = True) -> dict: ...
|
|
|
|
def entity_ensure(self, name: str, entity_type: str,
|
|
aliases: list = None, description: str = None) -> dict: ...
|
|
def entity_link(self, from_name: str, to_name: str,
|
|
relation: str, strength: float = 1.0) -> dict: ...
|
|
def entity_query(self, name: str = None, entity_type: str = None,
|
|
limit: int = 10) -> list: ...
|
|
|
|
def timeline_add(self, event_type: str, title: str,
|
|
description: str = None, importance: float = 0.5,
|
|
related_entities: list = None) -> dict: ...
|
|
def timeline_query(self, start: float = None, end: float = None,
|
|
event_type: str = None, limit: int = 20) -> list: ...
|
|
|
|
# ── Tier 3: Semantic ──
|
|
def semantic_index(self, text: str, source_type: str = "session",
|
|
session_id: str = None, message_id: int = None) -> dict: ...
|
|
def semantic_search(self, query: str, limit: int = 10,
|
|
min_score: float = 0.7) -> list: ...
|
|
def semantic_hybrid(self, query: str, limit: int = 10) -> list: ...
|
|
|
|
# ── Graph ──
|
|
def graph_traverse(self, start_entity: str, depth: int = 2,
|
|
relation_filter: str = None) -> list: ...
|
|
def graph_shortest_path(self, from_entity: str, to_entity: str) -> list: ...
|
|
def graph_central_entities(self, limit: int = 10) -> list: ...
|
|
def graph_communities(self) -> list: ...
|
|
def graph_rebuild(self) -> dict: ...
|
|
|
|
# ── Cross-Tier ──
|
|
def recall(self, query: str, tiers: list = None,
|
|
limit_per_tier: int = 5) -> dict: ...
|
|
def consolidate(self) -> dict: ...
|
|
def stats(self) -> dict: ...
|
|
```
|
|
|
|
### 8.2 Rückgabe-Format
|
|
|
|
```python
|
|
{
|
|
"success": True | False,
|
|
"data": <ergebnis>,
|
|
"tier": "tier2" | "tier3" | "graph" | "multi",
|
|
"meta": {
|
|
"query_time_ms": 42,
|
|
"results_count": 5,
|
|
"tiers_queried": ["tier2", "tier3"]
|
|
},
|
|
"error": None | "Fehlermeldung"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 9. Integration mit Hermes Agent
|
|
|
|
### 9.1 Memory Tool (tools/memory_tool.py)
|
|
|
|
```python
|
|
# Erweiterung um Tier-2/3-Aktionen
|
|
MEMORY_TOOL_SCHEMA = {
|
|
"name": "memory",
|
|
"parameters": {
|
|
"action": {
|
|
"type": "string",
|
|
"enum": [
|
|
# Tier 1 (bestehend)
|
|
"add", "replace", "remove", "read",
|
|
# Tier 2 (neu)
|
|
"fact_store", "fact_query", "fact_update", "fact_delete",
|
|
"entity_ensure", "entity_link", "entity_query",
|
|
"timeline_add", "timeline_query",
|
|
# Tier 3 (neu)
|
|
"semantic_search", "semantic_index",
|
|
# Graph (neu)
|
|
"graph_traverse", "graph_path", "graph_central",
|
|
# Cross-Tier (neu)
|
|
"recall", "consolidate", "stats"
|
|
]
|
|
},
|
|
# ... bestehende Parameter + neue
|
|
"category": {"type": "string"},
|
|
"confidence": {"type": "number"},
|
|
"entity_type": {"type": "string"},
|
|
"relation": {"type": "string"},
|
|
"depth": {"type": "integer", "default": 2},
|
|
"tiers": {"type": "array", "items": {"type": "string"}}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 9.2 Session Search Tool (tools/session_search_tool.py)
|
|
|
|
```python
|
|
# Erweiterung: Automatische Indexierung in Tier 3
|
|
# Nach jeder Session-Suche werden Top-Ergebnisse implizit in
|
|
# semantic_index gepusht (Hintergrund-Queue)
|
|
|
|
def _index_results_to_tier3(results: list, session_id: str):
|
|
"""Fire-and-forget: Indexiert Session-Ergebnisse für semantische Suche."""
|
|
for r in results:
|
|
api.semantic_index(
|
|
text=r["content"],
|
|
source_type="session",
|
|
session_id=session_id,
|
|
message_id=r.get("id")
|
|
)
|
|
```
|
|
|
|
### 9.3 Cronjob-Integration (cron/scheduler.py)
|
|
|
|
```python
|
|
# Neue Cron-Jobs für Memory-Maintenance
|
|
CRON_JOBS = {
|
|
"memory.consolidate": {
|
|
"schedule": "0 3 * * *", # Täglich 3 Uhr
|
|
"func": "hermes_memory.cron.consolidate.run",
|
|
"description": "Fakten deduplizieren & Konflikte auflösen"
|
|
},
|
|
"memory.embed_queue": {
|
|
"schedule": "*/5 * * * *", # Alle 5 Minuten
|
|
"func": "hermes_memory.cron.embed_queue.run",
|
|
"description": "Pending Embeddings verarbeiten"
|
|
},
|
|
"memory.graph_refresh": {
|
|
"schedule": "0 4 * * 0", # Sonntag 4 Uhr
|
|
"func": "hermes_memory.cron.graph_refresh.run",
|
|
"description": "Knowledge Graph neu aufbauen"
|
|
},
|
|
"memory.prune": {
|
|
"schedule": "0 2 1 * *", # Monatlich
|
|
"func": "hermes_memory.cron.prune.run",
|
|
"description": "Alte/archivierte Daten entfernen"
|
|
}
|
|
}
|
|
```
|
|
|
|
### 9.4 Skills-System-Integration
|
|
|
|
```python
|
|
# Skill: memory_recall
|
|
# Ermöglicht Skills, auf alle Tiers zuzugreifen
|
|
|
|
# In skill_manager_tool.py Erweiterung:
|
|
def skill_recall_context(skill_id: str, query: str) -> dict:
|
|
"""Liefert kontextuelle Informationen aus dem Memory für einen Skill."""
|
|
api = get_memory_api()
|
|
return api.recall(
|
|
query=query,
|
|
tiers=["tier1", "tier2", "tier3"],
|
|
limit_per_tier=3
|
|
)
|
|
|
|
# Skill-Manifest kann memory_tiers deklarieren:
|
|
SKILL_MANIFEST = {
|
|
"name": "project_tracker",
|
|
"memory_tiers": ["tier2", "tier3"],
|
|
"memory_queries": [
|
|
"aktuelle Projekte",
|
|
"offene Aufgaben",
|
|
"technische Entscheidungen"
|
|
]
|
|
}
|
|
```
|
|
|
|
### 9.5 System Prompt Integration
|
|
|
|
```python
|
|
# In agent_init.py / prompt_builder.py:
|
|
def build_memory_context(api: MemoryAPI) -> str:
|
|
"""Baut den Memory-Kontext für den System Prompt."""
|
|
parts = []
|
|
|
|
# Tier 1: Curated (bestehend, frozen snapshot)
|
|
parts.append(api.curated_get("memory"))
|
|
parts.append(api.curated_get("user"))
|
|
|
|
# Tier 2: Relevante Fakten (dynamisch, limitiert)
|
|
recent_facts = api.fact_query(
|
|
query="", category="user",
|
|
limit=5, min_confidence=0.8
|
|
)
|
|
parts.append("## Bekannte Fakten\n" + format_facts(recent_facts))
|
|
|
|
# Tier 2: Zentrale Entitäten
|
|
central = api.graph_central_entities(limit=5)
|
|
parts.append("## Wichtige Entitäten\n" + format_entities(central))
|
|
|
|
# Tier 3: Semantische Erinnerungen (letzte Session)
|
|
# Wird nicht in den Prompt injiziert, sondern über
|
|
# memory_manager.prefetch_all() nachgeladen
|
|
|
|
return "\n\n".join(parts)
|
|
```
|
|
|
|
---
|
|
|
|
## 10. Konfiguration
|
|
|
|
```python
|
|
# hermes_memory/config.py
|
|
|
|
DEFAULT_CONFIG = {
|
|
"profile": "default",
|
|
|
|
"tier2": {
|
|
"enabled": True,
|
|
"db_path": "{HERMES_HOME}/{profile}/memory/tier2.db",
|
|
"wal_mode": True,
|
|
"max_facts": 100_000,
|
|
"max_entities": 10_000,
|
|
"auto_dedupe": True
|
|
},
|
|
|
|
"tier3": {
|
|
"enabled": True,
|
|
"backend": "chroma", # "chroma" | "qdrant"
|
|
"path": "{HERMES_HOME}/{profile}/memory/tier3",
|
|
"embedding_model": "local",
|
|
"embedding_dim": 384,
|
|
"chunk_size": 512,
|
|
"chunk_overlap": 64,
|
|
"min_score": 0.7
|
|
},
|
|
|
|
"graph": {
|
|
"enabled": True,
|
|
"path": "{HERMES_HOME}/{profile}/memory/graph",
|
|
"auto_rebuild_interval_hours": 24,
|
|
"max_nodes": 50_000,
|
|
"centrality_algorithm": "betweenness" # "betweenness" | "pagerank" | "degree"
|
|
},
|
|
|
|
"cron": {
|
|
"consolidate_schedule": "0 3 * * *",
|
|
"embed_schedule": "*/5 * * * *",
|
|
"graph_rebuild_schedule": "0 4 * * 0",
|
|
"prune_schedule": "0 2 1 * *"
|
|
},
|
|
|
|
"limits": {
|
|
"fact_ttl_days": 365,
|
|
"session_index_max_age_days": 90,
|
|
"max_embedding_queue": 1000
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 11. Datenfluss-Diagramme
|
|
|
|
### 11.1 Schreib-Fluss (Session → Memory)
|
|
|
|
```
|
|
User Message
|
|
│
|
|
▼
|
|
┌─────────────┐
|
|
│ Agent Loop │
|
|
└──────┬──────┘
|
|
│
|
|
├──────────────────────────────┐
|
|
│ │
|
|
▼ ▼
|
|
┌─────────────┐ ┌─────────────────┐
|
|
│ Tier 1 │ │ Tier 2 │
|
|
│ memory_tool │ │ fact_store() │
|
|
│ (manuel) │ │ entity_ensure() │
|
|
└─────────────┘ │ timeline_add() │
|
|
└────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐
|
|
│ Embedding Queue │
|
|
│ (SQLite-Table) │
|
|
└────────┬────────┘
|
|
│
|
|
┌─────────────────┼─────────────────┐
|
|
│ │ │
|
|
▼ ▼ ▼
|
|
┌──────────┐ ┌──────────────┐ ┌──────────┐
|
|
│ Tier 3 │ │ Graph │ │ Cronjob │
|
|
│ semantic │ │ entity_link()│ │ consolidate
|
|
│ _index() │ │ graph_rebuild│ │ prune │
|
|
└──────────┘ └──────────────┘ └──────────┘
|
|
```
|
|
|
|
### 11.2 Lese-Fluss (Recall → Agent)
|
|
|
|
```
|
|
User Query
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────┐
|
|
│ memory(action="recall", query=..., tiers=[])│
|
|
└─────────────────────────────────────────────┘
|
|
│
|
|
├──────────┬──────────┬──────────┐
|
|
│ │ │ │
|
|
▼ ▼ ▼ ▼
|
|
┌───────┐ ┌────────┐ ┌─────────┐ ┌───────┐
|
|
│Tier 1 │ │ Tier 2 │ │ Tier 3 │ │ Graph │
|
|
│curated│ │ facts │ │semantic │ │traverse│
|
|
│_get() │ │_query()│ │_search()│ │_path() │
|
|
└───┬───┘ └───┬────┘ └────┬────┘ └───┬───┘
|
|
│ │ │ │
|
|
└─────────┴─────┬─────┴──────────┘
|
|
│
|
|
▼
|
|
┌─────────────┐
|
|
│ Merge & │
|
|
│ Rerank │
|
|
│ (Cross-Tier)│
|
|
└──────┬──────┘
|
|
│
|
|
▼
|
|
┌─────────────┐
|
|
│ System Prompt│
|
|
│ Injection │
|
|
└─────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 12. Sicherheit & Isolation
|
|
|
|
| Aspekt │ Maßnahme
|
|
| Profil-Isolation │ Jedes Profil hat eigene DBs & Vektor-Store
|
|
| Content-Scan │ threat_patterns.py wird auf alle Tier-2-Inhalte angewendet
|
|
| Injection-Guard │ §-Delimiter-Validierung für Tier 1 bleibt bestehen
|
|
| Zugriffskontrolle │ MemoryAPI prüft tool_call_id gegen session_id
|
|
| Audit-Log │ Alle Schreiboperationen in `memory_audit_log`-Tabelle
|
|
| Deduplizierung │ SHA-256-Hashing verhindert doppelte Fakten
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS memory_audit_log (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
timestamp REAL NOT NULL,
|
|
action TEXT NOT NULL,
|
|
tier TEXT NOT NULL,
|
|
actor TEXT NOT NULL, -- session_id │ cron │ user │ tool_name
|
|
target_uuid TEXT,
|
|
diff TEXT, -- JSON: {"old": ..., "new": ...}
|
|
success INTEGER DEFAULT 1
|
|
);
|
|
```
|
|
|
|
---
|
|
|
|
## 13. Performance-Ziele
|
|
|
|
| Operation │ Ziel-Latenz │ Skalierung
|
|
| Tier-2 Faktensuche (FTS5) │ < 50ms │ 100k Fakten
|
|
| Tier-3 Semantische Suche │ < 100ms │ 50k Chunks
|
|
| Graph Traversal (depth=3) │ < 30ms │ 10k Nodes
|
|
| Embedding (lokal, CPU) │ < 200ms/Chunk │ Batch-Verarbeitung
|
|
| Gesamt-Recall (3 Tiers) │ < 300ms │ Parallel-Queries
|
|
|
|
---
|
|
|
|
## 14. Migrationspfad
|
|
|
|
```
|
|
Phase 1: Tier 2 (SQLite)
|
|
├── Schema erstellen
|
|
├── Bestehende MEMORY.md parsen → facts
|
|
├── Integration memory_tool.py
|
|
└── Release v0.16.0
|
|
|
|
Phase 2: Tier 3 (Chroma/Qdrant)
|
|
├── Backend-Abstraktion
|
|
├── Embedding-Queue + Cronjob
|
|
├── Session-Search-Integration
|
|
└── Release v0.17.0
|
|
|
|
Phase 3: Graph (NetworkX)
|
|
├── Entity-Extraktion aus Sessions
|
|
├── Graph-Builder
|
|
├── Traversal-Tools
|
|
└── Release v0.18.0
|
|
|
|
Phase 4: Unified API & Skills
|
|
├── Cross-Tier Recall
|
|
├── Skill-Memory-Adapter
|
|
├── Performance-Optimierung
|
|
└── Release v1.0.0
|
|
```
|
|
|
|
---
|
|
|
|
## 15. Abhängigkeiten
|
|
|
|
```toml
|
|
[project.dependencies]
|
|
# Core (bereits in Hermes)
|
|
sqlite3 = "builtin"
|
|
|
|
# Tier 3
|
|
chromadb = { version = "^0.5.0", optional = true }
|
|
qdrant-client = { version = "^1.9.0", optional = true }
|
|
|
|
# Embeddings (lokal)
|
|
sentence-transformers = { version = "^3.0.0", optional = true }
|
|
|
|
# Graph
|
|
networkx = { version = "^3.3", optional = true }
|
|
|
|
# Utilities
|
|
numpy = "^1.26"
|
|
```
|
|
|
|
---
|
|
|
|
## Anhang A: Schnittstellen-Definitionen (Python)
|
|
|
|
### A.1 Tier 2 Interface
|
|
|
|
```python
|
|
# hermes_memory/tier2/facts.py
|
|
|
|
from dataclasses import dataclass
|
|
from typing import Optional
|
|
|
|
@dataclass
|
|
class Fact:
|
|
uuid: str
|
|
content: str
|
|
category: str
|
|
confidence: float
|
|
source_type: str
|
|
source_id: Optional[str]
|
|
created_at: float
|
|
updated_at: float
|
|
expires_at: Optional[float]
|
|
access_count: int
|
|
is_archived: bool
|
|
|
|
class FactStore:
|
|
def __init__(self, conn: sqlite3.Connection): ...
|
|
|
|
def store(self, content: str, category: str = "general",
|
|
confidence: float = 1.0, source_type: str = "user",
|
|
source_id: str = None) -> Fact: ...
|
|
|
|
def query(self, query: str = None, category: str = None,
|
|
limit: int = 10, min_confidence: float = 0.5,
|
|
fts: bool = True) -> list[Fact]: ...
|
|
|
|
def get_by_hash(self, content_hash: str) -> Optional[Fact]: ...
|
|
def get_by_uuid(self, uuid: str) -> Optional[Fact]: ...
|
|
def update(self, uuid: str, **fields) -> Fact: ...
|
|
def delete(self, uuid: str, soft: bool = True) -> bool: ...
|
|
def deduplicate(self) -> int: ... # Returns merged count
|
|
```
|
|
|
|
### A.2 Tier 3 Interface
|
|
|
|
```python
|
|
# hermes_memory/tier3/backend.py
|
|
|
|
from abc import ABC, abstractmethod
|
|
from dataclasses import dataclass
|
|
|
|
@dataclass
|
|
class SearchResult:
|
|
chunk_id: str
|
|
score: float
|
|
text: str
|
|
metadata: dict
|
|
|
|
class VectorBackend(ABC):
|
|
@abstractmethod
|
|
def index(self, chunks: list[str], payloads: list[dict]) -> list[str]: ...
|
|
|
|
@abstractmethod
|
|
def search(self, query_embedding: list[float], limit: int = 10,
|
|
filters: dict = None) -> list[SearchResult]: ...
|
|
|
|
@abstractmethod
|
|
def delete(self, chunk_ids: list[str]) -> bool: ...
|
|
|
|
@abstractmethod
|
|
def health(self) -> dict: ...
|
|
```
|
|
|
|
### A.3 Graph Interface
|
|
|
|
```python
|
|
# hermes_memory/graph/nx_store.py
|
|
|
|
import networkx as nx
|
|
|
|
class KnowledgeGraph:
|
|
def __init__(self, path: Path):
|
|
self.G = nx.DiGraph()
|
|
self.path = path
|
|
self._load()
|
|
|
|
def add_entity(self, uuid: str, name: str, entity_type: str,
|
|
**attrs) -> dict: ...
|
|
|
|
def add_relation(self, from_uuid: str, to_uuid: str,
|
|
relation_type: str, strength: float = 1.0,
|
|
**attrs) -> dict: ...
|
|
|
|
def traverse(self, start_uuid: str, depth: int = 2,
|
|
relation_filter: str = None) -> list[dict]: ...
|
|
|
|
def shortest_path(self, from_uuid: str, to_uuid: str) -> list[str]: ...
|
|
|
|
def centrality(self, algorithm: str = "betweenness",
|
|
limit: int = 10) -> list[dict]: ...
|
|
|
|
def communities(self, algorithm: str = "louvain") -> list[list[str]]: ...
|
|
|
|
def save(self) -> None: ...
|
|
def rebuild(self, tier2_conn: sqlite3.Connection) -> None: ...
|
|
```
|
|
|
|
---
|
|
|
|
*Ende der Architektur-Dokumentation*
|