Storage System Overview

Our intelligent storage system provides instant reads (RAM cache), durable persistence with smart adapter selection, cross-tab sync, and optional backend sync. Session 166.3 added BackendSyncEngine and syncToBackend() for fire-and-forget backend persistence.

Architecture Diagram

┌─────────────────────────────────────────────────────────────────┐
│ Frontend (Angular Apps)                                          │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │ GgStorage (Intelligent Storage)                          │   │
│  │ ├─ OPFS Adapter (large data, file-like)                   │   │
│  │ ├─ Dexie Adapter (IndexedDB, structured data)            │   │
│  │ └─ LocalStorage Adapter (small config data)              │   │
│  └──────────────────────────────────────────────────────────┘   │
│                         ↕️                                        │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │ BackendSyncEngine (Session 166.3) / SyncEngineService     │   │
│  │ - Fire-and-forget sync with retry queue                   │   │
│  │ - Offline-first: Local first, sync in background          │   │
│  └──────────────────────────────────────────────────────────┘   │
│                         ↕️                                        │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │ RealtimeSyncService (Socket.IO + Yjs)                     │   │
│  │ - Event broadcasting (universe, module, widget updates)   │   │
│  │ - CRDT collaborative editing (page content)              │   │
│  └──────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘
                            ↕️ HTTP/WebSocket
┌─────────────────────────────────────────────────────────────────┐
│ Backend API (NestJS)                                             │
│  MongoDB (Mongoose) | Socket.IO Server | Yjs WebSocket Server    │
└─────────────────────────────────────────────────────────────────┘

Intelligent Adapter Selection

Never manually specify adapters. The system analyzes data characteristics and chooses automatically:

Use semantic key names: auth/session/token, user/preferences, cache/api/response.

Real-Time Sync (Socket.IO + Yjs)

• Socket.IO — Event broadcasting (universe updates, module changes, widget updates)
• Yjs (CRDT) — Collaborative editing (page content, shared documents)

Collaborators see changes in ~15ms via Yjs; MongoDB persists in background (~100ms).

Performance Characteristics

Key: User experience is instant (~5ms) thanks to local-first architecture.

Session 166.3 Additions

• BackendSyncEngine — Generic fire-and-forget sync with retry queue
• syncToBackend() — One-liner companion for autoFlashState signals
• Two-signal pattern — Race-condition-free backend data merging
• StorageObservabilityService — Queue depth, quota, cache metrics

See BackendSyncEngine guide.

Core Features

• autoFlashState — Auto-persistent signals (IndexedDB, 0.3ms reads)
• BackendSyncEngine — Fire-and-forget backend sync with retry
• GgStorage — Intelligent adapter selection (OPFS, Dexie, localStorage)
• Cross-tab sync — Automatic state propagation
• 20GB+ capacity — OPFS + IndexedDB + localStorage

Related

• Getting Started — 5-minute setup
• API Reference — Complete API
• Flash Persistence + Conversation Genome — Chat persistence architecture