Add AI Council architecture: Tier 2/3/Graph implementation + Integration Plan
Architecture (Agent 1):
- hermes_memory/tier2/{schema,facts,entities,relations,timeline}.py
- hermes_memory/tier3/{backend,chroma_backend,embedder}.py
- hermes_memory/graph/nx_store.py
- hermes_memory/api/memory_api.py (unified API)
- hermes_memory/cron/{consolidate,embed_queue,graph_refresh,prune}.py
- hermes_memory/config.py + pyproject.toml
Integration Plan (Agent 3):
- INTEGRATION_PLAN.md: Memory Provider Plugin strategy
- Hermes Core needs minimal changes
- sync_turn() + prefetch() hooks
- Skills integration via nextlevel_search/remember
Auto-Extraction (Agent 2):
- ARCHITECTURE.md: Full extraction pipeline docs
- Chunking, Pre-Filter, LLM Prompts, Classification
- Entity-Linking, Temporal Reasoning, Deduplication
All files: Python syntax checked, ECC standards applied.
This commit is contained in:
+882
@@ -0,0 +1,882 @@
|
||||
# Hermes Memory Next Level — Technische Architektur
|
||||
|
||||
**Version:** 1.0.0 │ **Autor:** Architektur-Experte │ **Datum:** 2026-06-03
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
Hermes Memory Next Level (HMNL) ist ein mehrschichtiges, lokal laufendes Memory-Upgrade für Hermes Agent. Es erweitert das bestehende Key-Value Memory (Tier 1) um eine relationale Wissensbasis (Tier 2, SQLite) und eine semantische Vektorsuche (Tier 3, Qdrant/Chroma) mit Graph-Reasoning (NetworkX). Alle Tiers sind optional aktivierbar, lokal betreibbar und cloud-unabhängig.
|
||||
|
||||
---
|
||||
|
||||
## 2. Design-Prinzipien (ECC-Standard)
|
||||
|
||||
| Prinzip │ Beschreibung
|
||||
| Einfachheit │ Jedes Tier kann standalone betrieben werden
|
||||
| Kompaktheit │ SQLite-Tabellen mit │-Trennern, minimale Spaltenzahl
|
||||
| Erweiterbarkeit │ Plugin-Architektur für neue Memory-Provider
|
||||
| Lokalisierung │ Keine Cloud-Abhängigkeit, alles on-premise
|
||||
| Konsistenz │ Einheitliche API über alle Tiers hinweg
|
||||
|
||||
---
|
||||
|
||||
## 3. Tier-Architektur
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ TIER 1 — Curated Memory (Bestehend) │
|
||||
│ MEMORY.md │ USER.md │ §-delimited │ Frozen Snapshot │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ TIER 2 — Structured Knowledge (Neu) │
|
||||
│ SQLite │ Fakten │ Entitäten │ Relationen │ Zeitachse │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ TIER 3 — Semantic Memory (Neu) │
|
||||
│ Qdrant/Chroma │ Embeddings │ Ähnlichkeitssuche │ Cluster │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ GRAPH — Knowledge Graph (Neu) │
|
||||
│ NetworkX │ Entitäten als Nodes │ Relationen als Edges │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ API — Unified Memory Interface │
|
||||
│ Python-API │ Tool-Integration │ Cronjob │ Skills │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Modul-Struktur
|
||||
|
||||
```
|
||||
hermes_memory/
|
||||
│
|
||||
├── __init__.py # Public API exports
|
||||
├── config.py # Konfiguration & Defaults
|
||||
│
|
||||
├── tier1/ # Curated Memory (Wrapper)
|
||||
│ ├── __init__.py
|
||||
│ ├── curated_store.py # MEMORY.md / USER.md Interface
|
||||
│ └── snapshot.py # Frozen Snapshot Management
|
||||
│
|
||||
├── tier2/ # Structured Knowledge (SQLite)
|
||||
│ ├── __init__.py
|
||||
│ ├── schema.py # DB-Schema & Migrationen
|
||||
│ ├── connection.py # Pool & WAL-Handling
|
||||
│ ├── facts.py # CRUD für Fakten
|
||||
│ ├── entities.py # Entitäts-Verwaltung
|
||||
│ ├── relations.py # Relationen-Management
|
||||
│ ├── timeline.py # Zeitachsen-Queries
|
||||
│ └── search.py # FTS5 & strukturierte Suche
|
||||
│
|
||||
├── tier3/ # Semantic Memory (Vektor-DB)
|
||||
│ ├── __init__.py
|
||||
│ ├── backend.py # Abstrakte Backend-Schnittstelle
|
||||
│ ├── qdrant_backend.py # Qdrant-Implementierung
|
||||
│ ├── chroma_backend.py # Chroma-Implementierung
|
||||
│ ├── embedder.py # Embedding-Model Wrapper
|
||||
│ ├── chunks.py # Text-Chunking-Strategien
|
||||
│ └── semantic_search.py # Vektor-Suche & Reranking
|
||||
│
|
||||
├── graph/ # Knowledge Graph (NetworkX)
|
||||
│ ├── __init__.py
|
||||
│ ├── builder.py # Graph aus Tier 2 & 3 aufbauen
|
||||
│ ├── nx_store.py # NetworkX Persistenz (GraphML)
|
||||
│ ├── traversal.py # Pathfinding & Traversal
|
||||
│ ├── centrality.py # Wichtige Knoten identifizieren
|
||||
│ └── communities.py # Community Detection
|
||||
│
|
||||
├── api/ # Unified Interface
|
||||
│ ├── __init__.py
|
||||
│ ├── memory_api.py # Haupt-API-Klasse
|
||||
│ ├── tool_adapter.py # Integration memory_tool.py
|
||||
│ ├── session_adapter.py # Integration session_search_tool.py
|
||||
│ ├── cron_adapter.py # Integration cron/scheduler.py
|
||||
│ └── skill_adapter.py # Integration skills_system
|
||||
│
|
||||
├── cron/ # Hintergrund-Jobs
|
||||
│ ├── __init__.py
|
||||
│ ├── consolidate.py # Fakten-Deduplizierung
|
||||
│ ├── embed_queue.py # Embedding-Job-Queue
|
||||
│ ├── graph_refresh.py # Graph-Rebuild
|
||||
│ └── prune.py # Alte Daten ausdünnen
|
||||
│
|
||||
└── utils/
|
||||
├── __init__.py
|
||||
├── validators.py # Eingabe-Validierung
|
||||
├── sanitizers.py # Content-Sanitization
|
||||
└── hashing.py # Content-Hashing für Deduplizierung
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Datenbank-Schema (Tier 2 — SQLite)
|
||||
|
||||
### 5.1 Fakten-Tabelle
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS facts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
uuid TEXT NOT NULL UNIQUE, -- Global eindeutige ID
|
||||
content TEXT NOT NULL, -- Fakt als natürlicher Text
|
||||
content_hash TEXT NOT NULL, -- SHA-256 für Deduplizierung
|
||||
category TEXT, -- user │ project │ domain │ tool
|
||||
confidence REAL DEFAULT 1.0, -- 0.0 .. 1.0
|
||||
source_type TEXT NOT NULL, -- session │ memory │ tool │ cron │ user
|
||||
source_id TEXT, -- session_id │ tool_name │ NULL
|
||||
created_at REAL NOT NULL, -- Unix-Timestamp
|
||||
updated_at REAL NOT NULL, -- Unix-Timestamp
|
||||
expires_at REAL, -- TTL (optional)
|
||||
access_count INTEGER DEFAULT 0, -- Nutzungshäufigkeit
|
||||
last_accessed REAL, -- Letzter Zugriff
|
||||
is_archived INTEGER DEFAULT 0 -- Soft-Delete
|
||||
);
|
||||
|
||||
CREATE INDEX idx_facts_category ON facts(category);
|
||||
CREATE INDEX idx_facts_source ON facts(source_type, source_id);
|
||||
CREATE INDEX idx_facts_created ON facts(created_at DESC);
|
||||
CREATE INDEX idx_facts_hash ON facts(content_hash);
|
||||
CREATE INDEX idx_facts_confidence ON facts(confidence DESC);
|
||||
```
|
||||
|
||||
### 5.2 Entitäten-Tabelle
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS entities (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
uuid TEXT NOT NULL UNIQUE,
|
||||
name TEXT NOT NULL, -- Kanonischer Name
|
||||
aliases TEXT, -- JSON-Array: ["Alias1", "Alias2"]
|
||||
entity_type TEXT NOT NULL, -- person │ project │ tech │ org │ concept │ place
|
||||
description TEXT, -- Kurzbeschreibung
|
||||
first_seen REAL NOT NULL, -- Erstes Vorkommen
|
||||
last_seen REAL NOT NULL, -- Letztes Vorkommen
|
||||
occurrence_count INTEGER DEFAULT 1, -- Häufigkeit
|
||||
metadata TEXT -- JSON: {"key": "value"}
|
||||
);
|
||||
|
||||
CREATE INDEX idx_entities_name ON entities(name);
|
||||
CREATE INDEX idx_entities_type ON entities(entity_type);
|
||||
CREATE INDEX idx_entities_aliases ON entities(aliases); -- FTS5 für Aliase
|
||||
```
|
||||
|
||||
### 5.3 Relationen-Tabelle
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS relations (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
uuid TEXT NOT NULL UNIQUE,
|
||||
from_entity_id TEXT NOT NULL REFERENCES entities(uuid),
|
||||
to_entity_id TEXT NOT NULL REFERENCES entities(uuid),
|
||||
relation_type TEXT NOT NULL, -- works_on │ knows │ depends_on │ part_of │ related_to
|
||||
strength REAL DEFAULT 1.0, -- 0.0 .. 1.0
|
||||
evidence_fact_id TEXT REFERENCES facts(uuid), -- Begründender Fakt
|
||||
created_at REAL NOT NULL,
|
||||
updated_at REAL NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX idx_relations_from ON relations(from_entity_id);
|
||||
CREATE INDEX idx_relations_to ON relations(to_entity_id);
|
||||
CREATE INDEX idx_relations_type ON relations(relation_type);
|
||||
```
|
||||
|
||||
### 5.4 Timeline / Ereignisse
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS timeline (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
uuid TEXT NOT NULL UNIQUE,
|
||||
event_type TEXT NOT NULL, -- milestone │ decision │ error │ insight │ change
|
||||
title TEXT NOT NULL,
|
||||
description TEXT,
|
||||
related_entities TEXT, -- JSON-Array von entity_uuids
|
||||
related_facts TEXT, -- JSON-Array von fact_uuids
|
||||
session_id TEXT, -- Herkunft
|
||||
timestamp REAL NOT NULL,
|
||||
importance REAL DEFAULT 0.5 -- 0.0 .. 1.0
|
||||
);
|
||||
|
||||
CREATE INDEX idx_timeline_time ON timeline(timestamp DESC);
|
||||
CREATE INDEX idx_timeline_type ON timeline(event_type);
|
||||
```
|
||||
|
||||
### 5.5 FTS5 für Volltextsuche
|
||||
|
||||
```sql
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS facts_fts USING fts5(
|
||||
content,
|
||||
content_rowid='id',
|
||||
tokenize='unicode61'
|
||||
);
|
||||
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS entities_fts USING fts5(
|
||||
name || ' ' || COALESCE(description, ''),
|
||||
content_rowid='id',
|
||||
tokenize='unicode61'
|
||||
);
|
||||
```
|
||||
|
||||
### 5.6 Schema-Versionierung
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS memory_schema_version (
|
||||
version INTEGER NOT NULL,
|
||||
applied_at REAL NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Tier 3 — Vektor-DB Schema (Qdrant / Chroma)
|
||||
|
||||
### 6.1 Qdrant Collections
|
||||
|
||||
```python
|
||||
# Collection: memory_chunks
|
||||
{
|
||||
"name": "memory_chunks",
|
||||
"vectors": {
|
||||
"size": 384, # all-MiniLM-L6-v2 oder local embedding
|
||||
"distance": "Cosine"
|
||||
},
|
||||
"payload_schema": {
|
||||
"chunk_id": {"type": "keyword"},
|
||||
"fact_id": {"type": "keyword"}, # NULL wenn direkt aus Session
|
||||
"session_id": {"type": "keyword"},
|
||||
"message_id": {"type": "integer"}, # messages.id aus SQLite
|
||||
"source_type": {"type": "keyword"}, # fact │ session │ memory │ tool
|
||||
"category": {"type": "keyword"},
|
||||
"timestamp": {"type": "float"},
|
||||
"content_hash": {"type": "keyword"},
|
||||
"text_preview": {"type": "text"} # Erste 200 Zeichen
|
||||
}
|
||||
}
|
||||
|
||||
# Collection: entity_embeddings
|
||||
{
|
||||
"name": "entity_embeddings",
|
||||
"vectors": {
|
||||
"size": 384,
|
||||
"distance": "Cosine"
|
||||
},
|
||||
"payload_schema": {
|
||||
"entity_id": {"type": "keyword"},
|
||||
"entity_name": {"type": "keyword"},
|
||||
"entity_type": {"type": "keyword"},
|
||||
"description": {"type": "text"}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 6.2 Chroma Collections (Alternative)
|
||||
|
||||
```python
|
||||
# Chroma-Äquivalent
|
||||
client.create_collection(
|
||||
name="memory_chunks",
|
||||
metadata={"hnsw:space": "cosine"}
|
||||
)
|
||||
client.create_collection(
|
||||
name="entity_embeddings",
|
||||
metadata={"hnsw:space": "cosine"}
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Graph-Schema (NetworkX)
|
||||
|
||||
### 7.1 Node-Attribute
|
||||
|
||||
```python
|
||||
{
|
||||
"node_type": "entity", # entity │ fact │ session │ concept
|
||||
"uuid": "ent-uuid",
|
||||
"name": "Projekt Alpha",
|
||||
"entity_type": "project", # nur bei entity-Nodes
|
||||
"weight": 1.0, # Centrality / Wichtigkeit
|
||||
"created_at": 1717420800.0,
|
||||
"last_seen": 1717420800.0,
|
||||
"metadata": {} # Zusätzliche Attribute
|
||||
}
|
||||
```
|
||||
|
||||
### 7.2 Edge-Attribute
|
||||
|
||||
```python
|
||||
{
|
||||
"relation_type": "depends_on", # aus relations-Tabelle
|
||||
"strength": 0.85, # Gewicht
|
||||
"evidence": "fact-uuid", # Begründung
|
||||
"first_seen": 1717420800.0,
|
||||
"last_seen": 1717420800.0,
|
||||
"bidirectional": False
|
||||
}
|
||||
```
|
||||
|
||||
### 7.3 Persistenz
|
||||
|
||||
```python
|
||||
# Speicherung als GraphML (XML-basiert, menschenlesbar)
|
||||
nx.write_graphml(G, path / "knowledge_graph.graphml")
|
||||
|
||||
# Oder als Pickle für Performance
|
||||
nx.write_gpickle(G, path / "knowledge_graph.gpickle")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. API-Design (Unified Memory API)
|
||||
|
||||
### 8.1 Hauptklasse: MemoryAPI
|
||||
|
||||
```python
|
||||
class MemoryAPI:
|
||||
"""Unified interface for all memory tiers."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
profile: str = "default",
|
||||
tier2_enabled: bool = True,
|
||||
tier3_enabled: bool = True,
|
||||
graph_enabled: bool = True,
|
||||
tier3_backend: str = "chroma", # "qdrant" | "chroma"
|
||||
embedding_model: str = "local", # "local" | "openai" | "sentence-transformers"
|
||||
):
|
||||
...
|
||||
|
||||
# ── Tier 1: Curated ──
|
||||
def curated_get(self, store: str = "memory") -> str: ...
|
||||
def curated_add(self, content: str, store: str = "memory") -> dict: ...
|
||||
def curated_replace(self, old: str, new: str, store: str = "memory") -> dict: ...
|
||||
def curated_remove(self, substring: str, store: str = "memory") -> dict: ...
|
||||
|
||||
# ── Tier 2: Structured ──
|
||||
def fact_store(self, content: str, category: str = "general",
|
||||
confidence: float = 1.0, source: str = "user") -> dict: ...
|
||||
def fact_query(self, query: str, category: str = None,
|
||||
limit: int = 10, min_confidence: float = 0.5) -> list: ...
|
||||
def fact_get(self, uuid: str) -> dict: ...
|
||||
def fact_update(self, uuid: str, **fields) -> dict: ...
|
||||
def fact_delete(self, uuid: str, soft: bool = True) -> dict: ...
|
||||
|
||||
def entity_ensure(self, name: str, entity_type: str,
|
||||
aliases: list = None, description: str = None) -> dict: ...
|
||||
def entity_link(self, from_name: str, to_name: str,
|
||||
relation: str, strength: float = 1.0) -> dict: ...
|
||||
def entity_query(self, name: str = None, entity_type: str = None,
|
||||
limit: int = 10) -> list: ...
|
||||
|
||||
def timeline_add(self, event_type: str, title: str,
|
||||
description: str = None, importance: float = 0.5,
|
||||
related_entities: list = None) -> dict: ...
|
||||
def timeline_query(self, start: float = None, end: float = None,
|
||||
event_type: str = None, limit: int = 20) -> list: ...
|
||||
|
||||
# ── Tier 3: Semantic ──
|
||||
def semantic_index(self, text: str, source_type: str = "session",
|
||||
session_id: str = None, message_id: int = None) -> dict: ...
|
||||
def semantic_search(self, query: str, limit: int = 10,
|
||||
min_score: float = 0.7) -> list: ...
|
||||
def semantic_hybrid(self, query: str, limit: int = 10) -> list: ...
|
||||
|
||||
# ── Graph ──
|
||||
def graph_traverse(self, start_entity: str, depth: int = 2,
|
||||
relation_filter: str = None) -> list: ...
|
||||
def graph_shortest_path(self, from_entity: str, to_entity: str) -> list: ...
|
||||
def graph_central_entities(self, limit: int = 10) -> list: ...
|
||||
def graph_communities(self) -> list: ...
|
||||
def graph_rebuild(self) -> dict: ...
|
||||
|
||||
# ── Cross-Tier ──
|
||||
def recall(self, query: str, tiers: list = None,
|
||||
limit_per_tier: int = 5) -> dict: ...
|
||||
def consolidate(self) -> dict: ...
|
||||
def stats(self) -> dict: ...
|
||||
```
|
||||
|
||||
### 8.2 Rückgabe-Format
|
||||
|
||||
```python
|
||||
{
|
||||
"success": True | False,
|
||||
"data": <ergebnis>,
|
||||
"tier": "tier2" | "tier3" | "graph" | "multi",
|
||||
"meta": {
|
||||
"query_time_ms": 42,
|
||||
"results_count": 5,
|
||||
"tiers_queried": ["tier2", "tier3"]
|
||||
},
|
||||
"error": None | "Fehlermeldung"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Integration mit Hermes Agent
|
||||
|
||||
### 9.1 Memory Tool (tools/memory_tool.py)
|
||||
|
||||
```python
|
||||
# Erweiterung um Tier-2/3-Aktionen
|
||||
MEMORY_TOOL_SCHEMA = {
|
||||
"name": "memory",
|
||||
"parameters": {
|
||||
"action": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
# Tier 1 (bestehend)
|
||||
"add", "replace", "remove", "read",
|
||||
# Tier 2 (neu)
|
||||
"fact_store", "fact_query", "fact_update", "fact_delete",
|
||||
"entity_ensure", "entity_link", "entity_query",
|
||||
"timeline_add", "timeline_query",
|
||||
# Tier 3 (neu)
|
||||
"semantic_search", "semantic_index",
|
||||
# Graph (neu)
|
||||
"graph_traverse", "graph_path", "graph_central",
|
||||
# Cross-Tier (neu)
|
||||
"recall", "consolidate", "stats"
|
||||
]
|
||||
},
|
||||
# ... bestehende Parameter + neue
|
||||
"category": {"type": "string"},
|
||||
"confidence": {"type": "number"},
|
||||
"entity_type": {"type": "string"},
|
||||
"relation": {"type": "string"},
|
||||
"depth": {"type": "integer", "default": 2},
|
||||
"tiers": {"type": "array", "items": {"type": "string"}}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 9.2 Session Search Tool (tools/session_search_tool.py)
|
||||
|
||||
```python
|
||||
# Erweiterung: Automatische Indexierung in Tier 3
|
||||
# Nach jeder Session-Suche werden Top-Ergebnisse implizit in
|
||||
# semantic_index gepusht (Hintergrund-Queue)
|
||||
|
||||
def _index_results_to_tier3(results: list, session_id: str):
|
||||
"""Fire-and-forget: Indexiert Session-Ergebnisse für semantische Suche."""
|
||||
for r in results:
|
||||
api.semantic_index(
|
||||
text=r["content"],
|
||||
source_type="session",
|
||||
session_id=session_id,
|
||||
message_id=r.get("id")
|
||||
)
|
||||
```
|
||||
|
||||
### 9.3 Cronjob-Integration (cron/scheduler.py)
|
||||
|
||||
```python
|
||||
# Neue Cron-Jobs für Memory-Maintenance
|
||||
CRON_JOBS = {
|
||||
"memory.consolidate": {
|
||||
"schedule": "0 3 * * *", # Täglich 3 Uhr
|
||||
"func": "hermes_memory.cron.consolidate.run",
|
||||
"description": "Fakten deduplizieren & Konflikte auflösen"
|
||||
},
|
||||
"memory.embed_queue": {
|
||||
"schedule": "*/5 * * * *", # Alle 5 Minuten
|
||||
"func": "hermes_memory.cron.embed_queue.run",
|
||||
"description": "Pending Embeddings verarbeiten"
|
||||
},
|
||||
"memory.graph_refresh": {
|
||||
"schedule": "0 4 * * 0", # Sonntag 4 Uhr
|
||||
"func": "hermes_memory.cron.graph_refresh.run",
|
||||
"description": "Knowledge Graph neu aufbauen"
|
||||
},
|
||||
"memory.prune": {
|
||||
"schedule": "0 2 1 * *", # Monatlich
|
||||
"func": "hermes_memory.cron.prune.run",
|
||||
"description": "Alte/archivierte Daten entfernen"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 9.4 Skills-System-Integration
|
||||
|
||||
```python
|
||||
# Skill: memory_recall
|
||||
# Ermöglicht Skills, auf alle Tiers zuzugreifen
|
||||
|
||||
# In skill_manager_tool.py Erweiterung:
|
||||
def skill_recall_context(skill_id: str, query: str) -> dict:
|
||||
"""Liefert kontextuelle Informationen aus dem Memory für einen Skill."""
|
||||
api = get_memory_api()
|
||||
return api.recall(
|
||||
query=query,
|
||||
tiers=["tier1", "tier2", "tier3"],
|
||||
limit_per_tier=3
|
||||
)
|
||||
|
||||
# Skill-Manifest kann memory_tiers deklarieren:
|
||||
SKILL_MANIFEST = {
|
||||
"name": "project_tracker",
|
||||
"memory_tiers": ["tier2", "tier3"],
|
||||
"memory_queries": [
|
||||
"aktuelle Projekte",
|
||||
"offene Aufgaben",
|
||||
"technische Entscheidungen"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 9.5 System Prompt Integration
|
||||
|
||||
```python
|
||||
# In agent_init.py / prompt_builder.py:
|
||||
def build_memory_context(api: MemoryAPI) -> str:
|
||||
"""Baut den Memory-Kontext für den System Prompt."""
|
||||
parts = []
|
||||
|
||||
# Tier 1: Curated (bestehend, frozen snapshot)
|
||||
parts.append(api.curated_get("memory"))
|
||||
parts.append(api.curated_get("user"))
|
||||
|
||||
# Tier 2: Relevante Fakten (dynamisch, limitiert)
|
||||
recent_facts = api.fact_query(
|
||||
query="", category="user",
|
||||
limit=5, min_confidence=0.8
|
||||
)
|
||||
parts.append("## Bekannte Fakten\n" + format_facts(recent_facts))
|
||||
|
||||
# Tier 2: Zentrale Entitäten
|
||||
central = api.graph_central_entities(limit=5)
|
||||
parts.append("## Wichtige Entitäten\n" + format_entities(central))
|
||||
|
||||
# Tier 3: Semantische Erinnerungen (letzte Session)
|
||||
# Wird nicht in den Prompt injiziert, sondern über
|
||||
# memory_manager.prefetch_all() nachgeladen
|
||||
|
||||
return "\n\n".join(parts)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Konfiguration
|
||||
|
||||
```python
|
||||
# hermes_memory/config.py
|
||||
|
||||
DEFAULT_CONFIG = {
|
||||
"profile": "default",
|
||||
|
||||
"tier2": {
|
||||
"enabled": True,
|
||||
"db_path": "{HERMES_HOME}/{profile}/memory/tier2.db",
|
||||
"wal_mode": True,
|
||||
"max_facts": 100_000,
|
||||
"max_entities": 10_000,
|
||||
"auto_dedupe": True
|
||||
},
|
||||
|
||||
"tier3": {
|
||||
"enabled": True,
|
||||
"backend": "chroma", # "chroma" | "qdrant"
|
||||
"path": "{HERMES_HOME}/{profile}/memory/tier3",
|
||||
"embedding_model": "local",
|
||||
"embedding_dim": 384,
|
||||
"chunk_size": 512,
|
||||
"chunk_overlap": 64,
|
||||
"min_score": 0.7
|
||||
},
|
||||
|
||||
"graph": {
|
||||
"enabled": True,
|
||||
"path": "{HERMES_HOME}/{profile}/memory/graph",
|
||||
"auto_rebuild_interval_hours": 24,
|
||||
"max_nodes": 50_000,
|
||||
"centrality_algorithm": "betweenness" # "betweenness" | "pagerank" | "degree"
|
||||
},
|
||||
|
||||
"cron": {
|
||||
"consolidate_schedule": "0 3 * * *",
|
||||
"embed_schedule": "*/5 * * * *",
|
||||
"graph_rebuild_schedule": "0 4 * * 0",
|
||||
"prune_schedule": "0 2 1 * *"
|
||||
},
|
||||
|
||||
"limits": {
|
||||
"fact_ttl_days": 365,
|
||||
"session_index_max_age_days": 90,
|
||||
"max_embedding_queue": 1000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Datenfluss-Diagramme
|
||||
|
||||
### 11.1 Schreib-Fluss (Session → Memory)
|
||||
|
||||
```
|
||||
User Message
|
||||
│
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ Agent Loop │
|
||||
└──────┬──────┘
|
||||
│
|
||||
├──────────────────────────────┐
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────┐ ┌─────────────────┐
|
||||
│ Tier 1 │ │ Tier 2 │
|
||||
│ memory_tool │ │ fact_store() │
|
||||
│ (manuel) │ │ entity_ensure() │
|
||||
└─────────────┘ │ timeline_add() │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Embedding Queue │
|
||||
│ (SQLite-Table) │
|
||||
└────────┬────────┘
|
||||
│
|
||||
┌─────────────────┼─────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────────┐ ┌──────────┐
|
||||
│ Tier 3 │ │ Graph │ │ Cronjob │
|
||||
│ semantic │ │ entity_link()│ │ consolidate
|
||||
│ _index() │ │ graph_rebuild│ │ prune │
|
||||
└──────────┘ └──────────────┘ └──────────┘
|
||||
```
|
||||
|
||||
### 11.2 Lese-Fluss (Recall → Agent)
|
||||
|
||||
```
|
||||
User Query
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ memory(action="recall", query=..., tiers=[])│
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
├──────────┬──────────┬──────────┐
|
||||
│ │ │ │
|
||||
▼ ▼ ▼ ▼
|
||||
┌───────┐ ┌────────┐ ┌─────────┐ ┌───────┐
|
||||
│Tier 1 │ │ Tier 2 │ │ Tier 3 │ │ Graph │
|
||||
│curated│ │ facts │ │semantic │ │traverse│
|
||||
│_get() │ │_query()│ │_search()│ │_path() │
|
||||
└───┬───┘ └───┬────┘ └────┬────┘ └───┬───┘
|
||||
│ │ │ │
|
||||
└─────────┴─────┬─────┴──────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ Merge & │
|
||||
│ Rerank │
|
||||
│ (Cross-Tier)│
|
||||
└──────┬──────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ System Prompt│
|
||||
│ Injection │
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. Sicherheit & Isolation
|
||||
|
||||
| Aspekt │ Maßnahme
|
||||
| Profil-Isolation │ Jedes Profil hat eigene DBs & Vektor-Store
|
||||
| Content-Scan │ threat_patterns.py wird auf alle Tier-2-Inhalte angewendet
|
||||
| Injection-Guard │ §-Delimiter-Validierung für Tier 1 bleibt bestehen
|
||||
| Zugriffskontrolle │ MemoryAPI prüft tool_call_id gegen session_id
|
||||
| Audit-Log │ Alle Schreiboperationen in `memory_audit_log`-Tabelle
|
||||
| Deduplizierung │ SHA-256-Hashing verhindert doppelte Fakten
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS memory_audit_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp REAL NOT NULL,
|
||||
action TEXT NOT NULL,
|
||||
tier TEXT NOT NULL,
|
||||
actor TEXT NOT NULL, -- session_id │ cron │ user │ tool_name
|
||||
target_uuid TEXT,
|
||||
diff TEXT, -- JSON: {"old": ..., "new": ...}
|
||||
success INTEGER DEFAULT 1
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. Performance-Ziele
|
||||
|
||||
| Operation │ Ziel-Latenz │ Skalierung
|
||||
| Tier-2 Faktensuche (FTS5) │ < 50ms │ 100k Fakten
|
||||
| Tier-3 Semantische Suche │ < 100ms │ 50k Chunks
|
||||
| Graph Traversal (depth=3) │ < 30ms │ 10k Nodes
|
||||
| Embedding (lokal, CPU) │ < 200ms/Chunk │ Batch-Verarbeitung
|
||||
| Gesamt-Recall (3 Tiers) │ < 300ms │ Parallel-Queries
|
||||
|
||||
---
|
||||
|
||||
## 14. Migrationspfad
|
||||
|
||||
```
|
||||
Phase 1: Tier 2 (SQLite)
|
||||
├── Schema erstellen
|
||||
├── Bestehende MEMORY.md parsen → facts
|
||||
├── Integration memory_tool.py
|
||||
└── Release v0.16.0
|
||||
|
||||
Phase 2: Tier 3 (Chroma/Qdrant)
|
||||
├── Backend-Abstraktion
|
||||
├── Embedding-Queue + Cronjob
|
||||
├── Session-Search-Integration
|
||||
└── Release v0.17.0
|
||||
|
||||
Phase 3: Graph (NetworkX)
|
||||
├── Entity-Extraktion aus Sessions
|
||||
├── Graph-Builder
|
||||
├── Traversal-Tools
|
||||
└── Release v0.18.0
|
||||
|
||||
Phase 4: Unified API & Skills
|
||||
├── Cross-Tier Recall
|
||||
├── Skill-Memory-Adapter
|
||||
├── Performance-Optimierung
|
||||
└── Release v1.0.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 15. Abhängigkeiten
|
||||
|
||||
```toml
|
||||
[project.dependencies]
|
||||
# Core (bereits in Hermes)
|
||||
sqlite3 = "builtin"
|
||||
|
||||
# Tier 3
|
||||
chromadb = { version = "^0.5.0", optional = true }
|
||||
qdrant-client = { version = "^1.9.0", optional = true }
|
||||
|
||||
# Embeddings (lokal)
|
||||
sentence-transformers = { version = "^3.0.0", optional = true }
|
||||
|
||||
# Graph
|
||||
networkx = { version = "^3.3", optional = true }
|
||||
|
||||
# Utilities
|
||||
numpy = "^1.26"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Anhang A: Schnittstellen-Definitionen (Python)
|
||||
|
||||
### A.1 Tier 2 Interface
|
||||
|
||||
```python
|
||||
# hermes_memory/tier2/facts.py
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
@dataclass
|
||||
class Fact:
|
||||
uuid: str
|
||||
content: str
|
||||
category: str
|
||||
confidence: float
|
||||
source_type: str
|
||||
source_id: Optional[str]
|
||||
created_at: float
|
||||
updated_at: float
|
||||
expires_at: Optional[float]
|
||||
access_count: int
|
||||
is_archived: bool
|
||||
|
||||
class FactStore:
|
||||
def __init__(self, conn: sqlite3.Connection): ...
|
||||
|
||||
def store(self, content: str, category: str = "general",
|
||||
confidence: float = 1.0, source_type: str = "user",
|
||||
source_id: str = None) -> Fact: ...
|
||||
|
||||
def query(self, query: str = None, category: str = None,
|
||||
limit: int = 10, min_confidence: float = 0.5,
|
||||
fts: bool = True) -> list[Fact]: ...
|
||||
|
||||
def get_by_hash(self, content_hash: str) -> Optional[Fact]: ...
|
||||
def get_by_uuid(self, uuid: str) -> Optional[Fact]: ...
|
||||
def update(self, uuid: str, **fields) -> Fact: ...
|
||||
def delete(self, uuid: str, soft: bool = True) -> bool: ...
|
||||
def deduplicate(self) -> int: ... # Returns merged count
|
||||
```
|
||||
|
||||
### A.2 Tier 3 Interface
|
||||
|
||||
```python
|
||||
# hermes_memory/tier3/backend.py
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
|
||||
@dataclass
|
||||
class SearchResult:
|
||||
chunk_id: str
|
||||
score: float
|
||||
text: str
|
||||
metadata: dict
|
||||
|
||||
class VectorBackend(ABC):
|
||||
@abstractmethod
|
||||
def index(self, chunks: list[str], payloads: list[dict]) -> list[str]: ...
|
||||
|
||||
@abstractmethod
|
||||
def search(self, query_embedding: list[float], limit: int = 10,
|
||||
filters: dict = None) -> list[SearchResult]: ...
|
||||
|
||||
@abstractmethod
|
||||
def delete(self, chunk_ids: list[str]) -> bool: ...
|
||||
|
||||
@abstractmethod
|
||||
def health(self) -> dict: ...
|
||||
```
|
||||
|
||||
### A.3 Graph Interface
|
||||
|
||||
```python
|
||||
# hermes_memory/graph/nx_store.py
|
||||
|
||||
import networkx as nx
|
||||
|
||||
class KnowledgeGraph:
|
||||
def __init__(self, path: Path):
|
||||
self.G = nx.DiGraph()
|
||||
self.path = path
|
||||
self._load()
|
||||
|
||||
def add_entity(self, uuid: str, name: str, entity_type: str,
|
||||
**attrs) -> dict: ...
|
||||
|
||||
def add_relation(self, from_uuid: str, to_uuid: str,
|
||||
relation_type: str, strength: float = 1.0,
|
||||
**attrs) -> dict: ...
|
||||
|
||||
def traverse(self, start_uuid: str, depth: int = 2,
|
||||
relation_filter: str = None) -> list[dict]: ...
|
||||
|
||||
def shortest_path(self, from_uuid: str, to_uuid: str) -> list[str]: ...
|
||||
|
||||
def centrality(self, algorithm: str = "betweenness",
|
||||
limit: int = 10) -> list[dict]: ...
|
||||
|
||||
def communities(self, algorithm: str = "louvain") -> list[list[str]]: ...
|
||||
|
||||
def save(self) -> None: ...
|
||||
def rebuild(self, tier2_conn: sqlite3.Connection) -> None: ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Ende der Architektur-Dokumentation*
|
||||
Reference in New Issue
Block a user