🚨 URGENT: Clear Corrupted Data & Fix Sync

🎯 The Problem

The sync queue in localStorage is corrupted beyond auto-repair. The operations have all fields as undefined, which means the serialization/deserialization is failing.

Additionally, MongoDB is not being updated - universeIds: [] remains empty even after creating universes.

✅ IMMEDIATE FIX REQUIRED

Step 1: Clear ALL Browser Data 🧹

In BOTH browsers, open console and run:

// 1. Clear localStorage
localStorage.clear();

// 2. Clear IndexedDB
(async () => {
  const dbs = await indexedDB.databases();
  for (const db of dbs) {
    indexedDB.deleteDatabase(db.name);
    console.log(`✅ Deleted database: ${db.name}`);
  }
  console.log('✅ All IndexedDB databases cleared');
})();

// 3. Verify it's empty
console.log('localStorage items:', Object.keys(localStorage).length);

Step 2: Clear MongoDB Space Records 🗑️

In MongoDB Atlas/Compass, delete these space documents:

// Delete BOTH of these spaces:
{ "_id": { "$oid": "69067c087ebb621cd9ff8c3f" } }
{ "_id": { "$oid": "6909075c33381d2fd1da604a" } }

Why? These spaces have universeIds: [] and outdated data. Fresh start needed.

Step 3: Reload Both Browsers 🔄

CMD/CTRL + Shift + R (hard reload in both browsers)

🔍 Root Cause Analysis

Issue #1: autoFlashState Serialization

The syncQueue uses autoFlashState which persists to localStorage:

private syncQueue = autoFlashState<SyncOperation[]>('sync-queue', []);

Problem: When complex objects with methods or circular references are serialized to localStorage, they can become corrupted:

// What gets saved (JSON.stringify)
localStorage.setItem('flash:sync-queue', JSON.stringify(operations));

// What gets loaded (JSON.parse)
const operations = JSON.parse(localStorage.getItem('flash:sync-queue'));

// Result: undefined fields if serialization fails!

Issue #2: Space Not Syncing to Backend

Looking at your MongoDB, both spaces show:

• universeIds: [] ← Empty!
• updatedAt timestamps are old

This means: The space updates are NOT reaching MongoDB!

Possible causes:

1. Corrupted operations blocking the sync
2. Space update operations being skipped
3. Backend endpoint not receiving the update

🔧 Proposed Fix: Safer Queue Persistence

Option A: Use GgStorage Instead of autoFlashState

Change:

// Before (using autoFlashState - localStorage)
private syncQueue = autoFlashState<SyncOperation[]>('sync-queue', []);

// After (using GgStorage - IndexedDB with better serialization)
private syncQueue = signal<SyncOperation[]>([]);

async initialize(config: Partial<SyncConfig>) {
  // Load queue from IndexedDB
  const savedQueue = await this.storage.get<SyncOperation[]>('sync-queue');
  if (savedQueue) {
    this.syncQueue.set(savedQueue);
  }
  // ...
}

private async queueOperation(operation: SyncOperation): Promise<void> {
  this.syncQueue.update(queue => [...queue, operation]);
  // Persist to IndexedDB
  await this.storage.put('sync-queue', this.syncQueue());
}

Benefits:

• ✅ IndexedDB handles complex objects better
• ✅ No localStorage 5MB limit
• ✅ Better serialization/deserialization
• ✅ Async, non-blocking

Option B: Add Validation After Deserialization

Add to autoCleanupCorruptedQueue():

private async autoCleanupCorruptedQueue(): Promise<void> {
  try {
    const queue = this.syncQueue();
    
    // ✅ NEW: Check if queue itself is corrupted
    if (!Array.isArray(queue)) {
      this.logger.warn('[SyncEngine] 🚨 Queue is not an array! Resetting...');
      this.syncQueue.set([]);
      return;
    }
    
    if (queue.length === 0) {
      this.logger.debug('[SyncEngine] ✅ Queue is empty');
      return;
    }

    // ✅ ENHANCED: Deep validation of each operation
    const validOperations = queue.filter((op: any) => {
      // Check if op is even an object
      if (!op || typeof op !== 'object') {
        this.logger.warn('[SyncEngine] 🧹 Removing non-object operation:', typeof op);
        return false;
      }

      // Check required fields
      const hasRequiredFields = 
        op.id && 
        op.operation && 
        op.entity && 
        op.key;

      if (!hasRequiredFields) {
        this.logger.warn('[SyncEngine] 🧹 Removing operation with undefined fields:', {
          id: op.id || 'undefined',
          operation: op.operation || 'undefined',
          entity: op.entity || 'undefined',
          key: op.key || 'undefined'
        });
        return false;
      }

      // ✅ NEW: Validate data field for entities that require it
      if (op.entity === 'universe' && op.operation !== 'delete') {
        if (!op.data || typeof op.data !== 'object') {
          this.logger.warn('[SyncEngine] 🧹 Removing universe operation without data');
          return false;
        }
      }

      return true;
    });

    const removedCount = queue.length - validOperations.length;
    
    if (removedCount > 0) {
      this.syncQueue.set(validOperations);
      // ✅ PERSIST to localStorage after cleanup
      this.logger.warn(`[SyncEngine] 🧹 Auto-cleaned ${removedCount} corrupted operations`);
    }
  } catch (error) {
    this.logger.error('[SyncEngine] Critical error in cleanup:', error);
    this.syncQueue.set([]);
  }
}

🧪 Testing After Fix

Step 1: Verify Clean Start

Console should show:

[SyncEngine] ✅ Queue is empty
[SyncEngine] 🔄 Starting sync...
[SpaceApiService] Getting current user space
[SpaceApiService] ✅ Space loaded: chrome-space-1
[SyncEngine] ✅ Backend space saved locally (first sync)
[SyncEngine] ✅ Sync complete

Step 2: Create Universe

1. Click "Create Universe"
2. Name it "test-universe-1"
3. Wait 5 seconds

Console should show:

[SpaceService] ⚡ Universe created (offline-first): universe_1234567890
[SpaceService] ⚡ Space updated with new universe: space_xxxxx
[SyncEngine] Queued operation: update universe_1234567890
[SyncEngine] Queued operation: update space_xxxxx
[SyncEngine] 🔄 Starting sync...
[SyncEngine] 📤 Pushing 2 operations to backend
[SyncEngine] 🔄 Syncing operation: create universe universe_1234567890
[SpaceApiService] Creating universe: test-universe-1
[SpaceApiService] ✅ Universe created: universe_1234567890
[SyncEngine] ✅ Universe created in backend
[SyncEngine] 🔄 Syncing operation: update space space_xxxxx
[SpaceApiService] Updating space: space_xxxxx
[SpaceApiService] ✅ Space updated
[SyncEngine] ✅ Space synced to backend
[SyncEngine] ✅ Sync complete

Step 3: Verify MongoDB

Check MongoDB - space document should now show:

{
  "_id": { "$oid": "..." },
  "universeIds": ["universe_1234567890"],  ← NOT EMPTY!
  "updatedAt": "2025-11-17T...",  ← NEW timestamp!
  ...
}

📊 Expected Results

🚀 Next Steps

1. USER: Clear browser data & MongoDB (steps above)
2. USER: Hard reload both browsers
3. USER: Create a test universe
4. USER: Share console output & MongoDB data
5. AI: Verify fix worked or implement Option A (GgStorage)

💡 Why This Happened

1. autoFlashState uses localStorage
2. localStorage only stores strings (JSON.stringify)
3. Complex objects with functions/symbols get corrupted
4. Corrupted queue blocks all syncs
5. MongoDB never updates because syncs fail

Solution: Either clear data or migrate to IndexedDB (GgStorage)

⚠️ CRITICAL

DO NOT PROCEED without clearing corrupted data first!

The system cannot self-heal from this level of corruption because the queue itself is malformed at the storage level.