MongoDB Atlas Research & Analysis

Comprehensive research on MongoDB Atlas architecture, scaling, and AI workload optimization

MongoDB Atlas Research & Analysis

Purpose: Document all research findings regarding MongoDB Atlas architecture decisions for GoGrab Space AI infrastructure.

📋 Executive Summary

This document contains the complete research and analysis performed to determine the optimal MongoDB Atlas architecture for GoGrab Space, with specific focus on AI workloads including vector search, embeddings, and agent learning systems.

Key Decisions Made

Decision	Choice	Confidence
Cluster Architecture	Single cluster with multiple databases	98%
Production Tier	M30 (from Flex)	100%
Search Nodes	Add when needed (not immediately)	95%
Database Separation	4 databases for different workloads	95%

🏗️ MongoDB Atlas Architecture Deep-Dive

How MongoDB Clusters Work

MongoDB Atlas clusters share resources at the node level:

┌─────────────────────────────────────────────────────────────────────────────┐
│                        MONGODB ATLAS CLUSTER                                │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                    SHARED RESOURCES (Per Node)                       │   │
│  │                                                                      │   │
│  │   ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐   │   │
│  │   │    CPU     │  │    RAM     │  │  Storage   │  │   IOPS     │   │   │
│  │   │  (shared)  │  │  (shared)  │  │  (shared)  │  │  (shared)  │   │   │
│  │   └────────────┘  └────────────┘  └────────────┘  └────────────┘   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                    │                                        │
│                 ALL DATABASES COMPETE FOR THESE RESOURCES                   │
│                                    │                                        │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐   │
│  │ gograb-api  │ │ai-agents│ │gograb-ai    │ │ gograb-analytics    │   │
│  │             │ │             │ │             │ │                     │   │
│  │ collections │ │ collections │ │ collections │ │ collections         │   │
│  │ indexes     │ │ indexes     │ │ indexes     │ │ indexes             │   │
│  │ working set │ │ working set │ │ working set │ │ working set         │   │
│  └─────────────┘ └─────────────┘ └─────────────┘ └─────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘

Resource Sharing Impact

Resource	How It's Shared	Impact Level
RAM (WiredTiger Cache)	Single cache for ALL databases	⚠️ HIGH
CPU	Single thread pool	⚠️ MEDIUM
Storage IOPS	Single disk subsystem	⚠️ MEDIUM
Network	Shared connection pool	🟡 LOW
Indexes	Each loads into RAM	⚠️ HIGH

The WiredTiger Cache Consideration

The WiredTiger storage engine maintains a cache that all databases share:

┌─────────────────────────────────────────────────────────────────┐
│                    RAM (e.g., 8GB on M30)                       │
│                                                                 │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │           WiredTiger Cache (~50% of RAM)                  │  │
│  │                                                           │  │
│  │   ┌─────────────┐  ┌─────────────────────────────────┐   │  │
│  │   │ Core Data   │  │ Vector Embeddings               │   │  │
│  │   │ (spaces,    │  │                                 │   │  │
│  │   │  widgets,   │  │ Current: 987KB (25 docs)        │   │  │
│  │   │  themes)    │  │ Future: Could grow to GB        │   │  │
│  │   │             │  │                                 │   │  │
│  │   └─────────────┘  └─────────────────────────────────┘   │  │
│  │                                                           │  │
│  │   If cache is full → DISK READS → 100x slower            │  │
│  └──────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

🔍 Cross-Database Query Analysis

Critical Finding: `$lookup` Limitations

MongoDB's $lookup aggregation stage does NOT work across databases:

// ❌ THIS DOES NOT WORK!
db.getSiblingDB("ai-agents").agent_decisions.aggregate([
  {
    $lookup: {
      from: "gograb-api.spaces",  // DIFFERENT DATABASE - FAILS!
      localField: "tenantId",
      foreignField: "_id",
      as: "space"
    }
  }
]);
// Error: $lookup from collection must be in the same database

Available Workarounds

Method	Performance	Complexity	Best For
Application-level joins	Good	Medium	Most cases ✅
Data denormalization	Best	Low	Read-heavy
Atlas Data Lake	Slow	High	Analytics only
Change Streams sync	Good	High	Real-time sync

Why This Matters for AI

Our AI system needs cross-database access:

Core Data Needed	AI Usage	Frequency
`spaces`	Context for AI responses	Every request
`tenants`	Multi-tenant isolation	Every request
`widgets`	Widget generation context	Moderate
`modules`	Module generation context	Moderate
`themes`	Theme understanding	Low
`users`	Personalization	Every request

Solution: Application-level joins with caching (same cluster, minimal latency)

📊 Cluster Tier Analysis

Flex Cluster (Current)

Factor	Details
Cost	~$35/month
Infrastructure	Shared hardware
Performance	Variable, unpredictable
Vector Indexes	Limited to 10
Search Nodes	❌ Not available
Sharding	❌ Not available
Production Use	❌ Not recommended

M10 Tier

Factor	Details
Cost	~$58/month
RAM	1.7 GB
vCPU	0.5 (burstable)
Performance	Burstable (may throttle)
Search Nodes	✅ Available
Production Use	⚠️ Low-traffic only

M30 Tier (Recommended for Production)

Factor	Details
Cost	~$320/month
RAM	8 GB
vCPU	2 (dedicated)
Storage	40 GB (auto-expand)
IOPS	4,200 read/write
Connections	3,000 max
Performance	Consistent, predictable
Search Nodes	✅ Available
Sharding	✅ Available
Production Use	✅ Recommended

🔎 Search Nodes Analysis

What Are Search Nodes?

Dedicated infrastructure for Atlas Search and Vector Search, separate from data nodes:

┌─────────────────────────────────────────────────────────────────────────┐
│                        CLUSTER WITH SEARCH NODES                        │
│                                                                         │
│   ┌─────────────────────────────────────────────────────────────────┐  │
│   │                      DATA NODES (M30)                           │  │
│   │   Primary ─────── Secondary ─────── Secondary                   │  │
│   │   • Regular CRUD operations                                     │  │
│   │   • Standard queries                                            │  │
│   │   • NOT used for vector search                                  │  │
│   └─────────────────────────────────────────────────────────────────┘  │
│                                                                         │
│   ┌─────────────────────────────────────────────────────────────────┐  │
│   │                    SEARCH NODES (S30)                           │  │
│   │   Search Node 1 ─────── Search Node 2                           │  │
│   │   • Vector search queries                                       │  │
│   │   • Atlas Search queries                                        │  │
│   │   • Completely isolated from data nodes                         │  │
│   └─────────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────┘

Search Nodes Pricing

Search Tier	vCPUs	RAM	Storage	Hourly	Monthly
S20	2	4 GB	100 GB	$0.13	~$95
S30	4	8 GB	200 GB	$0.24	~$175
S40	8	16 GB	380 GB	$0.46	~$335
S50	16	32 GB	760 GB	$0.91	~$665

Note: Minimum 2 Search Nodes required when enabled.

When to Add Search Nodes

Trigger	Threshold
Vector search latency	> 100ms consistently
Embeddings count	> 500K documents
Database operations slowing	During AI workloads
Memory pressure	> 80% utilization

🆚 Single Cluster vs Separate Clusters

Detailed Comparison

Factor	Same Cluster	Separate Clusters
Monthly Cost	$320 (M30)	$640+ (2× M30)
Cross-DB queries	✅ Fast (2-5ms)	⚠️ Slow (10-20ms)
Fault isolation	⚠️ Shared	✅ Independent
Scaling	Together	Independent
Management	Simple	Complex
Connection strings	1	2
Backups	Unified	Separate

Why We Chose Single Cluster

Cross-reference needs - AI needs tenant/space data constantly
Search Nodes - Solve isolation problem within single cluster
Cost efficiency - No need to double infrastructure
Simpler operations - One cluster to manage
Current scale - 25 embeddings, 987KB (nowhere near limits)

📈 Scaling Strategies (Within Single Cluster)

Vector Quantization

Reduces memory footprint dramatically:

Quantization	Memory Reduction	Accuracy Impact
float32 (default)	0%	Perfect
int8 (scalar)	75%	~1% loss
int1 (binary)	97%	~5% loss

Example Impact:

1 Million Embeddings (1536 dimensions each)

float32: 6.14 GB RAM needed
int8:    1.54 GB RAM needed (75% savings!)
int1:    192 MB RAM needed (97% savings!)

Sharding (M30+)

Distributes data across multiple shards while maintaining:

✅ Same connection string
✅ Same databases
✅ Cross-database access (application-level)
✅ Transparent query routing

Tiered Storage

HOT TIER (in RAM)     → Recent 30 days, high-value patterns
WARM TIER (SSD)       → Last 6 months, queried on demand  
COLD TIER (Archive)   → 1+ year, compliance retention

💰 Cost Optimization Path

Phase	Tier	Monthly Cost	When
Development	Flex	~$35	Now
Pre-launch	M10	~$58	Optional testing
Production Launch	M30	~$320	At launch
Growth	M30 + 2×S30	~$670	When needed
Scale	M50 + Sharding	~$1,500+	Future

📚 Sources & References

MongoDB Atlas Documentation: atlas.mongodb.com/docs
Vector Search Best Practices: mongodb.com/vector-search
Search Nodes Documentation: mongodb.com/search-nodes
Pricing Calculator: mongodb.com/pricing

🔗 Related Documentation

Architectural Changes - What changes and why
New Architecture - Target architecture design
Backend AI API Plan - Implementation plan

MongoDB Atlas Research & Analysis

📋 Executive Summary

Key Decisions Made

🏗️ MongoDB Atlas Architecture Deep-Dive

How MongoDB Clusters Work

Resource Sharing Impact

The WiredTiger Cache Consideration

🔍 Cross-Database Query Analysis

Critical Finding: $lookup Limitations

Available Workarounds

Why This Matters for AI

📊 Cluster Tier Analysis

Flex Cluster (Current)

M10 Tier

M30 Tier (Recommended for Production)

🔎 Search Nodes Analysis

What Are Search Nodes?

Search Nodes Pricing

When to Add Search Nodes

🆚 Single Cluster vs Separate Clusters

Detailed Comparison

Why We Chose Single Cluster

📈 Scaling Strategies (Within Single Cluster)

Vector Quantization

Sharding (M30+)

Tiered Storage

💰 Cost Optimization Path

📚 Sources & References

🔗 Related Documentation

Critical Finding: `$lookup` Limitations