Hub-and-Spoke Umbau: Multi-Tenant Zentrale + Satellite-Agent

- Backend: Customer/Satellite Models, customer_id auf Server/Job/Audit - Satellite-API: heartbeat, poll (atomares Claiming), logs, result, scan-result, health-report - Auth via X-Api-Key (SHA-256 gehasht) - Job-Queue: pending/claimed/running/success/failed + Stale-Janitor - Batch-Trigger: ein Job pro Server, Satellite arbeitet sequenziell ab - Credentials bleiben lokal: nur symbolische credential_ref zentral - Neues Paket satellite/: Pull-Loop, WinRM/SSH/CAU/Scanner, PyInstaller-tauglich - Frontend: Kunden-Switcher, Satelliten-View, Polling statt WebSocket - Entfernt: WebSocket/Socket.io, Redis, zentrale Credentials, JobRunner - Docs: README/AGENTS/PROMPT auf neue Architektur aktualisiert
2026-08-07 03:42:06 +00:00
parent b91dd66fee
commit cc4c3fcecb
72 changed files with 2759 additions and 1642 deletions
@@ -1,42 +1,61 @@
 # Insight Updater - Project Prompt

 ## Overview
-Build a self-hosted update orchestration tool for Windows (CAU/WSUS) and Linux servers via WinRM/SSH. Web UI to manage servers, trigger updates, monitor progress live via WebSocket.
+Zentrale, mandantenfaehige Update-Orchestrierung fuer Kundennetzwerke (MSP-Modell).
+Hub-and-Spoke: eine zentrale Instanz (Docker, bei Insight-IT) plus schlanke
+Satelliten pro Kunde (eine Binary, kein Docker beim Kunden noetig). Satelliten pollen
+Jobs von der Zentrale, fuehren Updates lokal im Kundennetz aus und melden Ergebnisse
+zurueck. Dashboard zeigt alles nach Kunde sortiert.

 ## Target Stack
- **Backend**: FastAPI + Python 3.11+, SQLAlchemy + SQLite/PostgreSQL, structlog, python-winrm, paramiko
- **Frontend**: Vue 3 + TypeScript + Vite, Pinia, VueUse, Tailwind CSS, Socket.io client
- **Infra**: Docker Compose (backend, frontend, db, redis), Traefik labels for reverse proxy
- **CI/CD**: Gitea Actions / Woodpecker CI for build & deploy to monitoring (10.0.2.105)
+- **Zentrale Backend**: FastAPI + Python 3.11+, SQLAlchemy + SQLite/PostgreSQL, structlog
+- **Zentrale Frontend**: Vue 3 + TypeScript + Vite, Pinia, Tailwind CSS
+- **Satellite**: Python 3.11+, pywinrm, asyncssh, httpx, PyInstaller One-File-Binary
+- **Infra**: Docker Compose (nur Zentrale), Traefik Labels
+- **CI/CD**: Gitea Actions / Woodpecker CI fuer Build + Deploy auf monitoring (10.0.2.105)
+
+## Architektur-Entscheidungen
+- **Pull-Modell**: Satelliten pollen (default 30s). Keine eingehenden Verbindungen beim
+  Kunden, kein VPN, keine Firewall-Ausnahmen - nur ausgehend 443 zur Zentrale.
+- **Keine Live-Daten**: Log-Batches statt WebSocket. Dashboard refresht alle 10s.
+- **Credentials lokal**: WinRM/SSH-Zugangsdaten nur auf dem Satellite (credentials.yaml).
+  Zentral gibt es nur symbolische `credential_ref`-Namen.
+- **Netz-Orchestrierung**: 1-2 Satelliten pro Kunde steuern alle Server des Kunden,
+  aehnlich wie CAU einen Cluster steuert. Batch-Trigger legt pro Server einen Job an,
+  der Satellite arbeitet sie sequenziell ab.
+- **Job-Claiming atomar**: zwei Satelliten eines Kunden bekommen nie denselben Job.

 ## Core Features
-1. **Server Inventory** - Add/edit/delete servers (Windows/WinRM, Linux/SSH, CAU-Cluster)
-2. **Live Update Streaming** - WebSocket log stream with progress, status per node
-3. **CAU Cluster Orchestration** - Trigger `Invoke-CauRun`, show per-node phases
-4. **Linux Patch Management** - `apt/dnf/yum update` via SSH with sudo
-5. **Audit Log** - Structured JSON logs: who, when, what server, outcome
-5. **Health Checks** - `/health` endpoint, WinRM/SSH connectivity test
-6. **LDAP-ready Auth** - JWT tokens, LDAP config schema prepared, local admin fallback
+1. **Kunden + Satelliten** - CRUD, API-Key einmalig angezeigt, rotierbar
+2. **Server-Inventar pro Kunde** - manuell oder per Netzwerk-Scan (Auto-Discovery)
+3. **Job-Queue** - pending/claimed/running/success/failed/cancelled + Stale-Janitor
+4. **Windows Update** - WinRM, Microsoft.Update.Session, optional Reboot
+5. **Linux Update** - apt/dnf/yum via SSH mit sudo, optional Reboot
+6. **CAU** - Invoke-CauRun auf Failover-Clustern
+7. **Netzwerk-Scan** - Ping + Port 5985/22, legt Hosts zentral als Server an
+8. **Audit-Log** - strukturiert, pro Kunde filterbar

 ## Non-Goals
- No WSUS/SCCM replacement, no approval workflows
- No agent deployment (agentless WinRM/SSH only)
- No multi-tenancy / RBAC beyond admin/user
+- Kein WSUS/SCCM-Ersatz, keine Approval-Workflows
+- Kein Live-Streaming (bewusst: Pull + Batches)
+- Kein RBAC ueber admin/user hinaus
+- Keine zentral gespeicherten Kunden-Credentials

 ## Success Criteria
- Add server → see "Online/Offline", last patch date
- Click "Update" → live WebSocket log stream → final status Success/Failed
- CAU: Trigger cluster update, see per-node Pre/Post/Reboot phases
- Linux: Add SSH creds, trigger update, see apt/dnf output
- `docker compose up -d` → all healthy in <5 min on fresh VM
- Deploy to monitoring (10.0.2.105) via `git push` + CI works
- LDAP config schema exists, service stub wired, functional later
+- Kunde anlegen -> Satellite anlegen -> API-Key einmalig angezeigt
+- Satellite startet -> erscheint als "online" im Dashboard (Heartbeat)
+- Netzwerk-Scan -> gefundene Hosts im Inventar (discovered_by_scan)
+- Update triggern -> Satellite claimed Job -> Logs + Ergebnis im Dashboard
+- Batch: "Alle Server updaten" -> ein Job pro Server, sequenzielle Abarbeitung
+- Zwei Satelliten eines Kunden: kein Job doppelt
+- Satellite offline waehrend Job -> Janitor markiert Job nach Timeout als failed
+- `docker compose up -d` -> Zentrale healthy in <5 min

 ## Verification Commands
 ```bash
 curl -f http://localhost:8000/health
-curl -f http://localhost:3000/    # frontend
-docker compose ps  # all healthy
+curl -f http://localhost:3000/
+docker compose ps
 ```

 ## Deployment Target
@@ -44,111 +63,50 @@ docker compose ps  # all healthy
 - **User**: b0rbor4d (sudo via Vaultwarden)
 - **Docker**: Podman/Docker Compose v2
 - **Reverse Proxy**: Traefik (labels on compose services)
+- **Erreichbarkeit**: HTTPS 443 von Kundenstandorten aus (ausgehend)
 - **Git Remote**: ssh://git@gitea.insight-it.de:2222/b0rbor4d/insight-updater.git

 ## Security
- Credentials encrypted at rest (Fernet/AES-GCM, key from env)
- WinRM: NTLM/Kerberos, HTTPS preferred, Cert validation configurable
- SSH: Key-based auth preferred, password fallback encrypted
- JWT: RS256, short expiry, refresh token rotation
- Audit log: immutable append-only (SQLite WAL / PG)
+- Dashboard: JWT RS256, kurze Expiry
+- Satelliten: API-Key (ius_...), SHA-256 gehasht in DB, Prefix fuer Anzeige
+- Keine Kunden-Credentials in der zentralen DB
+- Audit-Log: append-only
+- TLS: Traefik + LetsEncrypt

 ## Project Structure
 ```
 ~/projects/insight-updater/
-├── backend/
+├── backend/            # Zentrale API
 │   ├── app/
-│   │   ├── api/          # FastAPI routes
-│   │   ├── core/         # config, security, db
-│   │   ├── models/       # SQLAlchemy models
-│   │   ├── schemas/      # Pydantic schemas
-│   │   ├── services/     # business logic (winrm, ssh, cau, audit)
-│   │   ├── websocket/    # Socket.io / FastAPI WS handlers
+│   │   ├── api/routes/ # auth, customers, satellites, servers, updates, audit, satellite_api
+│   │   ├── core/       # config, security, db, logging, exceptions
+│   │   ├── models/     # Customer, Satellite, Server, UpdateJob, UpdateLog, AuditLog, User
+│   │   ├── schemas/    # Pydantic
+│   │   ├── services/   # audit, janitor
 │   │   └── main.py
-│   ├── tests/
 │   ├── Dockerfile
-│   ├── requirements.txt
 │   └── pyproject.toml
-├── frontend/
+├── frontend/           # Dashboard
 │   ├── src/
-│   │   ├── components/
-│   │   ├── views/
-│   │   ├── stores/
-│   │   ├── api/
-│   │   └── main.ts
+│   │   ├── views/      # Dashboard, Customers, Satellites, Servers, Updates, Audit, Login
+│   │   ├── stores/     # auth, customers, satellites, servers, updates
+│   │   └── components/ # AppLayout (mit Kunden-Switcher)
 │   ├── Dockerfile
-│   ├── package.json
-│   └── vite.config.ts
+│   └── package.json
+├── satellite/          # Remote-Agent
+│   ├── satellite/
+│   │   ├── runner.py   # Main-Loop
+│   │   ├── client.py   # Zentral-API-Client
+│   │   ├── config.py   # config.yaml + credentials.yaml
+│   │   ├── winrm_exec.py, ssh_exec.py, cau_exec.py, scanner.py
+│   ├── config.example.yaml
+│   ├── credentials.example.yaml
+│   └── pyproject.toml
 ├── docker-compose.yml
 ├── docker-compose.prod.yml
-├── .env.example
-├── .gitignore
-├── README.md
-├── AGENTS.md
-└── PROMPT.md
+└── .env.example
 ```

-## Key Libraries
- `fastapi`, `uvicorn`, `sqlalchemy[asyncio]`, `alembic`
- `python-winrm[kerberos]`, `paramiko`, `asyncssh`
- `python-socketio[asyncio]`, `redis`, `structlog`
- `cryptography`, `python-jose[cryptography]`, `passlib[bcrypt]`
- `pydantic-settings`, `pydantic[email]`
- `pytest`, `pytest-asyncio`, `httpx`
-
-## Environment Variables (.env.example)
-```env
-# Core
-APP_ENV=development
-SECRET_KEY=change-me-32-chars-min
-ENCRYPTION_KEY=change-me-32-chars-base64
-JWT_ALGORITHM=RS256
-JWT_PRIVATE_KEY_PATH=/app/keys/private.pem
-JWT_PUBLIC_KEY_PATH=/app/keys/public.pem
-
-# Database
-DATABASE_URL=sqlite+aiosqlite:///./data/app.db
-# DATABASE_URL=postgresql+asyncpg://user:pass@db:5432/updater
-
-# Redis
-REDIS_URL=redis://redis:6379/0
-
-# WinRM
-WINRM_TRANSPORT=ntlm
-WINRM_CERT_VALIDATION=ignore
-
-# LDAP (stub)
-LDAP_ENABLED=false
-LDAP_URI=ldaps://dc.insight.local:636
-LDAP_BIND_DN=CN=svc_updater,OU=Services,DC=insight,DC=local
-LDAP_BIND_PASSWORD=
-LDAP_USER_SEARCH_BASE=OU=Users,DC=insight,DC=local
-LDAP_USER_FILTER=(sAMAccountName={username})
-
-# Frontend
-VITE_API_URL=http://localhost:8000
-VITE_WS_URL=ws://localhost:8000
-```
-
-## Development Workflow
-```bash
-# Local dev
-cd backend && pip install -e . && uvicorn app.main:app --reload
-cd frontend && npm install && npm run dev
-
-# Docker dev
-docker compose up -d --build
-
-# Tests
-cd backend && pytest
-cd frontend && npm run test
-```
-
-## Remote Deploy (monitoring)
-```bash
-# On monitoring host
-git clone ssh://git@gitea.insight-it.de:2222/b0rbor4d/insight-updater.git
-cd insight-updater
-cp .env.example .env  # fill secrets
-docker compose -f docker-compose.prod.yml up -d --build
-```
+## Environment Variables
+Zentrale siehe `.env.example`: SECRET_KEY, DATABASE_URL, JWT-Keys, LDAP-Stub.
+Satellite siehe `satellite/config.example.yaml`: central_url, api_key, poll_interval.