Universe Database Architecture - Multi-Tenant Data Isolation
Architecture for automatic database creation and management per universe
🗄️ Universe Database Architecture
Critical Insight: Each universe needs its own database scope for data isolation and space-builder control
🎯 The Vision
Problem: Space-builders need to manage their users' data
Solution: Automatic database creation per universe with visual management
Key Principle: Visual/Perceptive + Real
- Space-builders SEE their database visually
- Data is REAL in MongoDB (multi-tenant isolated)
- Everything organized by universe
🏗️ Architecture Design
✅ APPROVED: Hybrid Architecture
Decision: Database per Space + Collections per Universe
Rationale:
- ✅ ONE database per space-builder (simple, manageable)
- ✅ Collections prefixed by universe (clear isolation)
- ✅ Shared assets at space level (module reusability)
- ✅ Fast cross-universe queries (analytics)
- ✅ Scales efficiently (100+ universes, 1 database)
Three-Tier Hybrid System
┌──────────────────────────────────────────────────────────────┐
│ TIER 1: Platform Database (gograb-admin scope) │
│ Database: gograb_platform │
├──────────────────────────────────────────────────────────────┤
│ Collection: spaces # All space-builders │
│ Collection: platform_users # Platform users │
│ Collection: subscriptions # Billing/plans │
│ Collection: platform_logs # Platform-wide logs │
│ │
│ Purpose: Platform-level management │
│ Access: GoGrab Admins only │
└──────────────────────────────────────────────────────────────┘
↓ creates
┌──────────────────────────────────────────────────────────────┐
│ TIER 2: Space Database (ONE per space-builder) │
│ Database: space_{spaceId} │
├──────────────────────────────────────────────────────────────┤
│ │
│ SHARED Collections (Space-Level - space_ prefix): │
│ ├── space_metadata # Space settings │
│ ├── team_members # Space team │
│ ├── billing # Space billing │
│ ├── space_modules # Space-level modules │
│ ├── space_widgets # Space-level widgets │
│ ├── space_flows # Space-level flows │
│ ├── space_templates # Space-level templates │
│ ├── space_pages # Reusable pages │
│ └── audit_logs # Space-wide logs │
│ │
│ UNIVERSE Collections (Prefixed by Universe): │
│ ├── universe_{id}_metadata # Universe settings │
│ ├── universe_{id}_pages # Pages from Editor │
│ ├── universe_{id}_widgets # Widget instances │
│ ├── universe_{id}_flows # Logic flows │
│ ├── universe_{id}_modules # Packaged modules │
│ ├── universe_{id}_deployments # Deployment history │
│ │ │
│ ├── universe_{id}_users # App users (if deployed) │
│ ├── universe_{id}_products # Products (e-commerce) │
│ ├── universe_{id}_orders # Orders (e-commerce) │
│ ├── universe_{id}_contacts # Contacts (CRM) │
│ └── universe_{id}_[custom] # Custom collections │
│ │
│ Purpose: Space-builder's complete ecosystem │
│ Access: Space-builder (owner) + team members │
└──────────────────────────────────────────────────────────────┘
Key Benefits of Hybrid
- Module Reusability ✅
Create widget ONCE in space_templates
↓
Use in universe_shop_pages
Use in universe_marketplace_pages
Use in universe_wholesale_pages
↓
Update once, updates everywhere!
- Clear Organization ✅
One database per space-builder:
├── Shared (reusable across universes)
└── Universes (isolated by prefix)
- Efficient Queries ✅
Query ONE database:
├── Get all orders across all universes (analytics)
├── Find shared widgets (fast)
└── Universe-specific queries (indexed by prefix)
- Simple Billing ✅
1 space-builder = 1 database = 1 bill
🔧 Implementation Strategy
Automatic Database Creation (Hybrid Model)
// When space-builder signs up:
1. Create Space
↓
2. Backend automatically creates:
- MongoDB database: `space_{spaceId}`
- Collections:
* space_metadata
* team_members
* space_templates
* space_flows
* space_templates
↓
3. Space-builder ready to create universes!
// When universe is created within space:
1. Space-builder creates universe in Space Map
↓
2. Backend automatically creates collections in space database:
- universe_{universeId}_metadata
- universe_{universeId}_pages
- universe_{universeId}_widgets
- universe_{universeId}_flows
- universe_{universeId}_modules
- universe_{universeId}_deployments
↓
3. GgStorage configured with space + universe scope
↓
4. Space-builder has immediate access to all their data
Storage System Integration (Hybrid)
// libs/storage-system enhanced with space + universe scoping
class GgStorage {
// Current implementation (works globally)
async put(key: string, value: any): Promise<void>
async get(key: string): Promise<any>
async list(options?: ListOptions): Promise<string[]>
// ✅ NEW: Space-scoped operations (shared assets)
async putInSpace(spaceId: string, key: string, value: any): Promise<void> {
const scopedKey = `space:${spaceId}:shared:${key}`;
return this.put(scopedKey, value);
// Syncs to MongoDB: space_{spaceId}.shared_* collections
}
async getFromSpace(spaceId: string, key: string): Promise<any> {
const scopedKey = `space:${spaceId}:shared:${key}`;
return this.get(scopedKey);
}
async listInSpace(spaceId: string, prefix?: string): Promise<string[]> {
const scopedPrefix = `space:${spaceId}:shared:${prefix || ''}`;
return this.list({ prefix: scopedPrefix });
}
// ✅ NEW: Universe-scoped operations (universe-specific)
async putInUniverse(spaceId: string, universeId: string, key: string, value: any): Promise<void> {
const scopedKey = `space:${spaceId}:universe:${universeId}:${key}`;
return this.put(scopedKey, value);
// Syncs to MongoDB: space_{spaceId}.universe_{universeId}_* collections
}
async getFromUniverse(spaceId: string, universeId: string, key: string): Promise<any> {
const scopedKey = `space:${spaceId}:universe:${universeId}:${key}`;
return this.get(scopedKey);
}
async listInUniverse(spaceId: string, universeId: string, prefix?: string): Promise<string[]> {
const scopedPrefix = `space:${spaceId}:universe:${universeId}:${prefix || ''}`;
return this.list({ prefix: scopedPrefix });
}
// ✅ NEW: Query universe data with filters
async queryUniverseData(
spaceId: string,
universeId: string,
collection: string,
filter: any
): Promise<any[]> {
const keys = await this.listInUniverse(spaceId, universeId, collection);
const docs = await Promise.all(keys.map(k => this.get(k)));
return this.applyFilter(docs, filter);
}
// ✅ NEW: Query across all universes in space (analytics)
async queryAcrossUniverses(
spaceId: string,
collection: string,
filter: any
): Promise<any[]> {
// Query all universe_{*}_{collection} in space database
const keys = await this.list({ prefix: `space:${spaceId}:universe:` });
const collectionKeys = keys.filter(k => k.includes(`:${collection}:`));
const docs = await Promise.all(collectionKeys.map(k => this.get(k)));
return this.applyFilter(docs, filter);
}
}
📊 Database Schema Per Universe
Auto-Created Collections
// Automatically created when universe is created
universe_{universeId}/
├── metadata # Universe settings
│ ├── _id: universeId
│ ├── name: "My E-Commerce Site"
│ ├── owner: userId
│ ├── created: Date
│ ├── deployed: boolean
│ ├── deploymentUrl: string
│ └── settings: {...}
│
├── pages # Page definitions from Editor
│ ├── {pageId}: {
│ │ name, layout, widgets[], flows[], ...
│ │ }
│ └── ...
│
├── widgets # Widget instances
│ ├── {widgetId}: {
│ │ type, properties, variant, ...
│ │ }
│ └── ...
│
├── flows # Logic flows from Singularity
│ ├── {flowId}: {
│ │ name, blocks[], connections[], ...
│ │ }
│ └── ...
│
├── modules # Packaged modules
│ ├── {moduleId}: {
│ │ name, manifest, bundle, ...
│ │ }
│ └── ...
│
└── deployments # Deployment history
├── {deploymentId}: {
│ timestamp, status, url, config, ...
│ }
└── ...
Space-Builder Defined Collections
// Created when space-builder deploys modules with data models
universe_{universeId}/
├── users # If auth module deployed
│ ├── {userId}: { email, name, role, ... }
│ └── ...
│
├── products # If e-commerce module deployed
│ ├── {productId}: { name, price, inventory, ... }
│ └── ...
│
├── orders # If e-commerce module deployed
│ ├── {orderId}: { userId, items[], total, ... }
│ └── ...
│
├── contacts # If CRM module deployed
│ ├── {contactId}: { name, email, company, ... }
│ └── ...
│
└── [custom_collection] # Custom collections from modules
└── ...
🎨 Space Management App Architecture
gograb-admin vs space-management
| Aspect | gograb-admin | space-management |
|---|---|---|
| Users | Platform admins | Space-builders |
| Purpose | Manage platform | Manage universes |
| Scope | ALL spaces, ALL users | Own spaces only |
| Data Access | Platform-wide | Universe-scoped |
| Features | User management, billing, platform health | Universe data, deployments, analytics |
Space Management App Structure
apps/space-management/
├── src/
│ ├── app/
│ │ ├── dashboard/ # Overview of all universes
│ │ ├── universe-selector/ # Switch between universes
│ │ ├── data-browser/ # Browse universe data
│ │ │ ├── collections-view/ # View all collections
│ │ │ ├── data-table/ # Table view of records
│ │ │ ├── data-editor/ # Edit individual records
│ │ │ └── query-builder/ # Visual query builder
│ │ ├── deployment-manager/ # Manage deployments
│ │ ├── analytics/ # Universe analytics
│ │ ├── users-manager/ # Manage app users
│ │ └── settings/ # Universe settings
│ └── ...
├── project.json
└── README.md
Key Features
1. Universe Dashboard
┌────────────────────────────────────────────┐
│ My Universes │
├────────────────────────────────────────────┤
│ ┌──────────────────────────────────────┐ │
│ │ 🌌 E-Commerce Site │ │
│ │ Deployed: my-shop.gograb.space │ │
│ │ Users: 1,234 | Orders: 456 │ │
│ │ [View Data] [Analytics] [Deploy] │ │
│ └──────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────┐ │
│ │ 🌌 CRM Platform │ │
│ │ Status: Development │ │
│ │ Contacts: 89 | Deals: 23 │ │
│ │ [View Data] [Analytics] [Deploy] │ │
│ └──────────────────────────────────────┘ │
└────────────────────────────────────────────┘
2. Data Browser (Critical for space-builders!)
┌────────────────────────────────────────────┐
│ E-Commerce Site > Data │
├────────────────────────────────────────────┤
│ Collections: │
│ ├─ 👥 users (1,234 records) │
│ ├─ 🛍️ products (156 records) │
│ ├─ 📦 orders (456 records) │
│ ├─ 💬 reviews (789 records) │
│ └─ 📊 analytics (12,345 events) │
│ │
│ Selected: orders │
│ ┌──────────────────────────────────────┐ │
│ │ Order ID | User | Total | Status │ │
│ │ ─────────────────────────────────────│ │
│ │ #12345 | john@... | $99.99 | Paid │ │
│ │ #12346 | jane@... | $49.99 | Paid │ │
│ │ [Edit] [Delete] [Export] │ │
│ └──────────────────────────────────────┘ │
└────────────────────────────────────────────┘
3. Query Builder
Visual MongoDB query builder:
- Select collection
- Add filters (visual)
- Preview results
- Export data
- Create reports
🤖 AI Enhancement Strategy
gograb-ai Library Review & Enhancement
Current State: Needs to be production-ready, not mock
Required Enhancements:
// libs/gograb-ai/src/lib/services/
1. ✅ ai-orchestrator.service.ts
- Real AI provider integration (OpenAI, Anthropic, local models)
- Multi-model support
- Token management
- Cost tracking
2. ✅ ai-code-generator.service.ts
- Generate real TypeScript code
- Generate component templates
- Generate service logic
- Generate test files
3. ✅ ai-flow-generator.service.ts
- Convert natural language → flows
- Use practical blocks (HTTP, Auth, etc.)
- Generate complete flows with error handling
- Optimize flow structure
4. ✅ ai-data-analyzer.service.ts
- Analyze user data patterns
- Suggest optimizations
- Detect anomalies
- Generate insights
5. ✅ ai-training-coordinator.service.ts (NEW!)
- Collect usage patterns
- Track successful flows
- Store training data
- Prepare for custom model training
AI Integration Points
1. Widget Builder
✅ "Generate widget from description"
→ AI creates complete widget module
2. Singularity
✅ "Create flow to send welcome email after signup"
→ AI generates flow with practical blocks
3. Editor
✅ "Design e-commerce product page"
→ AI creates page with widgets
4. Module Builder
✅ "Package complete CRM solution"
→ AI suggests which assets to include
5. Data Browser (NEW!)
✅ "Find all orders from last week over $100"
→ AI generates MongoDB query
📋 Implementation Roadmap
v1.1 - Database Management (Week 1-2)
Priority: CRITICAL
1. Universe Database Auto-Creation
- On universe create → auto-create MongoDB database
- Configure storage-system with universe scope
- Set up permissions (space-builder = admin)
2. Data Browser Component
- libs/data-browser-lib/ (NEW)
- View collections
- Browse records
- Edit data
- Query builder
3. Space Management App Scaffold
- apps/space-management/ (NEW)
- Universe selector
- Data browser integration
- Deployment manager
- Analytics dashboard
4. Storage System Enhancement
- Universe-scoped operations
- Collection management
- Query capabilities
- Export/import
Deliverable: Space-builders can view and manage their users' data
v1.2 - AI Integration (Week 3-4)
Priority: HIGH
1. gograb-ai Library Enhancement
- Real AI provider integration (OpenAI/Claude)
- Remove all mocks
- Production-ready services
- Cost tracking
2. AI Flow Generation
- Natural language → complete flows
- Uses practical blocks automatically
- Includes error handling
- Optimizes structure
3. AI Widget Generation
- Description → complete widget module
- Generates TypeScript + HTML + SCSS
- Follows platform-agnostic patterns
4. AI Data Insights
- Analyze universe data
- Suggest optimizations
- Detect issues
- Generate reports
5. Training Data Collection
- Track successful patterns
- Store for future model training
- Privacy-preserved
- Opt-in by space-builders
Deliverable: AI generates real, working code and flows
v1.3 - Performance & Polish (Week 5-6)
1. Bundle Optimization
- Tree-shaking improvements
- Code splitting
- Lazy loading modules
- Asset optimization
2. Caching Strategy
- Intelligent caching
- Service workers
- Offline capability
- Fast reloads
3. UI/UX Polish
- Loading state improvements
- Animation refinements
- Accessibility audit
- Mobile optimization
4. Documentation Videos
- Complete walkthrough
- Feature-specific tutorials
- Best practices
- Troubleshooting guides
Deliverable: Professional, polished experience
v2.0 - Module Marketplace (Month 2)
1. Module Marketplace
- Browse community modules
- Rate & review
- Install modules
- Publish modules
2. Flutter Export
- Generate Flutter code
- Cross-platform modules
- Mobile deployment
- Desktop support
3. Collaboration Features
- Multi-user editing
- Real-time sync
- Comments & annotations
- Version control
4. Enterprise Features
- Team management
- Role-based access
- Audit logs
- SSO integration
Deliverable: Complete ecosystem
🔄 Database Management Blocks
NEW Practical Blocks Needed
// Add to business-logic-block-templates.ts
✅ CREATE_COLLECTION
- Create new collection in universe database
- Define schema
- Set permissions
✅ QUERY_DATABASE
- Visual MongoDB query builder
- Filter, sort, limit
- Return results
✅ INSERT_RECORD
- Add new record to collection
- Validate against schema
- Return inserted ID
✅ UPDATE_RECORD
- Update existing record
- Partial or full update
- Return updated record
✅ DELETE_RECORD
- Remove record from collection
- Cascade deletes (optional)
- Return success
✅ AGGREGATE_DATA
- MongoDB aggregation pipeline
- Group, sum, average, etc.
- Return aggregated results
✅ EXPORT_COLLECTION
- Export to CSV/JSON/Excel
- Filter before export
- Download or email
These integrate with the space-management app's data browser!
🎯 User Experience Flow
Space-Builder Journey
1. Create Universe "My E-Commerce"
↓ (automatic)
Database created: universe_my_ecommerce
Collections: metadata, pages, widgets, flows, modules
2. Build Platform (Singularity + Editor + Modules)
↓
All assets stored in universe database
3. Deploy Platform
↓
Live at: my-shop.gograb.space
End-users can register, browse products, place orders
4. Manage Data (Space Management App)
↓
Open: /space-management/universe/my-ecommerce/data
↓
See collections:
- users (1,234 registered users)
- products (156 products)
- orders (456 orders)
- reviews (789 reviews)
↓
Actions:
- Browse records
- Edit individual records
- Run queries (find all orders > $100)
- Export data (CSV/Excel)
- View analytics
- Monitor activity
🔐 Security & Permissions
Data Access Control
Platform Level (Tier 1):
├── GoGrab Admins: Full access to everything
└── Space-builders: No access to platform data
Universe Level (Tier 2):
├── Space-builder (owner): Full access to their universe
├── Assigned editors: Edit pages/flows
└── Viewers: Read-only access
Application Level (Tier 3):
├── Space-builder: Full access to app data
├── App admins: Manage users
└── End-users: Own data only
💡 Why This is Critical
For Space-Builders
- ✅ See their users' data
- ✅ Manage records visually
- ✅ Run reports and analytics
- ✅ Export data anytime
- ✅ Control access
For End-Users
- ✅ Data properly isolated
- ✅ Fast access (scoped queries)
- ✅ Secure (multi-tenant)
- ✅ Scalable
For Platform
- ✅ Perfect multi-tenancy
- ✅ Clean data organization
- ✅ Easy to scale
- ✅ Manageable
🚀 Implementation Priority
Immediate (This completes the platform!)
- ✅ Practical blocks (DONE)
- ⏳ Universe database auto-creation
- ⏳ Storage-system universe scoping
- ⏳ Space-management app scaffold
- ⏳ Data browser component
Short Term
- gograb-ai real integration
- AI flow generation
- Training data collection
Medium Term
- Module marketplace
- Flutter export
- Collaboration features
This architecture completes the platform builder vision! 🎊
Next Steps:
- Implement universe-scoped storage
- Create space-management app
- Build data browser
- Enhance gograb-ai library
- Real AI integration throughout
The platform builds platforms with intelligent data management! 🚀