MongoDB Production Architecture

🎯 Overview

GoGrab Space uses a MongoDB-first architecture for all data persistence and real-time collaboration.

Stack Components:

• MongoDB (Mongoose) - Primary database
• Socket.IO - Real-time event broadcasting
• Yjs (CRDT) - Collaborative editing
• GgStorage - Client-side intelligent storage (OPFS/IndexedDB)

🏗️ Architecture Diagram

┌─────────────────────────────────────────────────────────────────┐
│ Frontend (Angular Apps)                                         │
│                                                                 │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │ GgStorage (Intelligent Storage)                          │  │
│  │ ├─ OPFS Adapter (large data, file-like)                │  │
│  │ ├─ Dexie Adapter (IndexedDB, structured data)          │  │
│  │ └─ LocalStorage Adapter (small config data)            │  │
│  └──────────────────────────────────────────────────────────┘  │
│                         ↕️                                       │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │ SyncEngineService                                        │  │
│  │ - Bidirectional sync: Local ↔ MongoDB                  │  │
│  │ - Conflict resolution: Timestamp-based                  │  │
│  │ - Offline-first: Always hits GgStorage first           │  │
│  └──────────────────────────────────────────────────────────┘  │
│                         ↕️                                       │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │ RealtimeSyncService                                      │  │
│  │ - Socket.IO client (real-time events)                   │  │
│  │ - Yjs provider (CRDT collaboration)                     │  │
│  │ - Cross-tab synchronization                             │  │
│  └──────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
                            ↕️ HTTP/WebSocket
┌─────────────────────────────────────────────────────────────────┐
│ Backend API (NestJS)                                            │
│                                                                 │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │ MongoDB (Mongoose)                                       │  │
│  │ ├─ Spaces Collection                                    │  │
│  │ ├─ Universes Collection                                 │  │
│  │ ├─ Modules Collection                                   │  │
│  │ ├─ Widgets Collection                                   │  │
│  │ └─ Assets Collection                                    │  │
│  └──────────────────────────────────────────────────────────┘  │
│                         ↕️                                       │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │ Real-time Services                                       │  │
│  │ ├─ Socket.IO Server (port 3000/sync)                   │  │
│  │ └─ Yjs WebSocket Server (port 3000/yjs)                │  │
│  └──────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

📦 Frontend Implementation

1. GgStorage (Client-Side Intelligent Storage)

Location: libs/libs-foundational/storage-system/src/core/engine/gg-storage.ts

Key Features:

• Intelligent Policy Engine - Auto-selects optimal adapter based on data characteristics
• Multi-Adapter Support - OPFS, IndexedDB, localStorage
• Flash Persistence - Cross-tab reactive updates
• Size-Based Routing - Large data to OPFS, structured to IndexedDB, small to localStorage

Usage:

import { GgStorage } from '@storage-system';

@Injectable()
export class MyService {
  private storage = inject(GgStorage);
  
  async saveUniverse(universe: Universe) {
    // 🧠 Storage system automatically chooses optimal adapter
    await this.storage.set(`universe_${universe.id}`, universe);
    
    // ✅ Flash Persistence: All tabs update instantly via onEvent
  }
  
  ngOnInit() {
    // Listen for real-time updates
    this.storage.onEvent((event) => {
      if (event.type === 'put' && event.key?.startsWith('universe_')) {
        console.log('Universe updated:', event.key);
        // Refresh UI
      }
    });
  }
}

See: Intelligent Adapter Selection

2. SyncEngineService (MongoDB Synchronization)

Location: libs/libs-foundational/core-angular/src/lib/services/sync-engine.service.ts

Key Features:

• Local-First - All operations hit GgStorage first (5-10ms)
• Background Sync - Non-blocking sync to MongoDB
• Conflict Resolution - Timestamp-based (configurable)
• Offline Tolerance - Works 100% offline
• Queue Management - Retry failed sync operations

Data Flow:

User Action → GgStorage (instant) → UI Update
                    ↓
            SyncEngineService (background)
                    ↓
            Backend API → MongoDB
                    ↓
            Socket.IO broadcast → Other clients

Usage:

import { SyncEngineService } from '@libs-foundational/core-angular';

@Injectable()
export class SpaceService {
  private syncEngine = inject(SyncEngineService);
  
  async createUniverse(universe: Universe) {
    // 1️⃣ Save locally first (instant)
    await this.storage.set(`universe_${universe.id}`, universe);
    
    // 2️⃣ Queue sync to MongoDB (background)
    this.syncEngine.queueSync({
      id: crypto.randomUUID(),
      operation: 'create',
      entity: 'universe',
      key: `universe_${universe.id}`,
      data: universe,
      timestamp: Date.now(),
      retries: 0
    });
    
    // ✅ User sees instant UI update, MongoDB syncs in background
  }
}

3. RealtimeSyncService (Socket.IO + Yjs)

Location: libs/libs-foundational/core-angular/src/lib/services/realtime-sync.service.ts

Key Features:

• Socket.IO Client - Real-time event broadcasting
• Yjs Provider - CRDT collaborative editing
• Cross-Browser Sync - Changes sync across all open tabs/devices
• Automatic Reconnection - Handles network interruptions

Usage:

import { RealtimeSyncService } from '@libs-foundational/core-angular';

@Injectable()
export class UniverseService {
  private realtimeSync = inject(RealtimeSyncService);
  
  async updateUniverse(universeId: string, updates: Partial<Universe>) {
    // 1️⃣ Update locally
    const universe = await this.storage.get(`universe_${universeId}`);
    const updated = { ...universe, ...updates, updatedAt: Date.now() };
    await this.storage.set(`universe_${universeId}`, updated);
    
    // 2️⃣ Broadcast to other clients
    this.realtimeSync.broadcastUniverseUpdate(universeId, updated);
    
    // ✅ All connected clients receive update instantly
  }
  
  ngOnInit() {
    // Listen for remote updates
    this.realtimeSync.onUniverseUpdate.subscribe((update) => {
      console.log('Remote universe update:', update);
      // Sync to local storage
      void this.syncRemoteToLocal(update);
    });
  }
}

🔧 Backend Implementation

1. MongoDB Connection

Location: apis/backend-api/src/app/app.module.ts

MongooseModule.forRootAsync({
  imports: [ConfigModule],
  useFactory: async (configService: ConfigService) => ({
    uri: configService.get<string>('MONGODB_CONNECTION_STRING'),
    dbName: configService.get<string>('MONGODB_DATABASE', 'gograb-api'),
  }),
  inject: [ConfigService],
})

Environment Variables:

MONGODB_CONNECTION_STRING=mongodb+srv://user:[email protected]/
MONGODB_DATABASE=gograb-api

2. MongoDB Schemas

Example: Space Schema

Location: apis/backend-api/src/app/spaces/spaces.schema.ts

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';
import { Document } from 'mongoose';

@Schema({ timestamps: true })
export class Space extends Document {
  @Prop({ required: true })
  spaceId: string;
  
  @Prop({ required: true })
  name: string;
  
  @Prop()
  description?: string;
  
  @Prop({ required: true })
  ownerId: string;
  
  @Prop({ type: Object })
  meta?: Record<string, any>;
  
  @Prop()
  createdAt: Date;
  
  @Prop()
  updatedAt: Date;
}

export const SpaceSchema = SchemaFactory.createForClass(Space);

3. Socket.IO Server

Location: apis/backend-api/src/app/realtime/realtime.gateway.ts

Features:

• Universe creation/update/delete events
• Module synchronization
• Widget updates
• Page content changes

Events:

// Server → Client
socket.emit('universe:created', { universeId, universe });
socket.emit('universe:updated', { universeId, changes });
socket.emit('module:installed', { universeId, moduleId });

// Client → Server
socket.emit('join:universe', { universeId });
socket.emit('leave:universe', { universeId });

4. Yjs WebSocket Server

Location: apis/backend-api/src/app/yjs/yjs-server.ts

Features:

• CRDT collaborative editing
• Conflict-free updates
• Automatic merging
• Persistence to MongoDB

Usage:

// Client connects to ws://localhost:3000/yjs/universe-123
const ydoc = new Y.Doc();
const provider = new WebsocketProvider(
  'ws://localhost:3000/yjs',
  'universe-123',
  ydoc
);

// Shared data structure
const ymap = ydoc.getMap('universe');
ymap.set('name', 'My Universe');

// ✅ Changes sync automatically via CRDT

🔄 Data Flow Examples

Example 1: Creating a Universe

1️⃣ User clicks "Create Universe"
   └─> UniverseCreationWizardComponent.createUniverse()

2️⃣ Save to GgStorage (instant, ~5ms)
   └─> storage.set('universe_123', universe)
   └─> UI updates immediately

3️⃣ Queue sync to MongoDB (background)
   └─> syncEngine.queueSync({ operation: 'create', entity: 'universe', ... })

4️⃣ Sync to MongoDB (~50-200ms)
   └─> Backend API: POST /api/universes
   └─> MongoDB: db.universes.insertOne(universe)

5️⃣ Broadcast via Socket.IO (~10ms)
   └─> socket.emit('universe:created', { universeId: '123', universe })

6️⃣ Other clients receive update
   └─> realtimeSync.onUniverseCreated.subscribe()
   └─> storage.set('universe_123', universe)
   └─> UI refreshes automatically

Total Time:

• User sees update: ~5ms (GgStorage)
• Other clients see update: ~260ms (GgStorage → MongoDB → Socket.IO → GgStorage)

Example 2: Editing Page Content (Collaborative)

1️⃣ User types in editor
   └─> Yjs provider captures changes

2️⃣ Yjs syncs to server via WebSocket (~10ms)
   └─> ws://localhost:3000/yjs/page-456

3️⃣ Server broadcasts to all connected clients (~5ms)
   └─> Other clients receive CRDT update

4️⃣ Yjs merges changes (conflict-free)
   └─> All editors show same content

5️⃣ Background: Yjs persistence saves to MongoDB (~100ms)
   └─> db.pages.updateOne({ id: '456' }, { $set: { content: ydoc } })

Total Time:

• Collaborators see changes: ~15ms (Yjs CRDT)
• MongoDB persisted: ~100ms (background)

📊 Performance Characteristics

Key Takeaways:

• ✅ User experience is instant (~5ms) thanks to local-first architecture
• ✅ Real-time sync is fast (~200-300ms for full round trip)
• ✅ Collaborative editing is near-instant (~15ms via Yjs CRDT)

🛡️ Conflict Resolution

Strategy: Last-Write-Wins (Timestamp-Based)

// SyncEngineService.resolveConflict()
private resolveConflict(local: any, remote: any): any {
  const strategy = this.config().conflictResolution;
  
  if (strategy === 'local-wins') return local;
  if (strategy === 'remote-wins') return remote;
  
  // Timestamp-based (default)
  const localTime = local.updatedAt || 0;
  const remoteTime = remote.updatedAt || 0;
  
  return remoteTime > localTime ? remote : local;
}

See: SyncEngineService.resolveConflict() for full implementation

🔗 Related Documentation

• Intelligent Adapter Selection - How GgStorage chooses adapters
• Real-time Sync Implementation - Socket.IO + Yjs details
• Storage Event System - Flash Persistence internals
• Backend API Architecture - NestJS modules

🚀 Getting Started

1. Start MongoDB

# Local MongoDB
docker run -d -p 27017:27017 --name gograb-mongo mongo:latest

# Or use MongoDB Atlas (recommended for production)

2. Configure Environment

# apis/backend-api/.env.local
MONGODB_CONNECTION_STRING=mongodb://localhost:27017
MONGODB_DATABASE=gograb-api

3. Start Backend API

yarn serve:backend
# Runs on http://localhost:3000

4. Start Frontend

yarn dev:gograb
# Runs on http://localhost:5200

5. Verify Connection

• Open browser console
• Check for: ✅ MongoDB connected in backend logs
• Check for: ✅ Socket.IO connected in frontend console

✅ Production Checklist

• MongoDB connection configured
• Socket.IO server running
• Yjs server running
• GgStorage initialized
• SyncEngineService enabled
• RealtimeSyncService connected
• Cross-tab sync working
• Conflict resolution tested
• Offline mode functional
• Real-time collaboration tested

Last Updated: November 19, 2025 (Session 11.25.2.2)
Status: ✅ Production-Ready
Stack: MongoDB + Socket.IO + Yjs + GgStorage