S282.3 Infrastructure Rename Handoff

Documentation for Sessions

S282.3 Handoff: Infrastructure DNS Rename — sandbox.gograb.space → ax42-01.gograb.space

Session: 282.3 | Date: 2026-03-28 | Branch: session-280-285 Status: Complete — runtime code, server-side, Cloudflare, and Railway all updated

1. What Happened & Why

We operate two Hetzner dedicated servers. The old server (AX42-U, 144.76.69.55) was historically named sandbox.gograb.space. The new server (AX102-U, 178.63.165.83) follows the convention ax102-01.gograb.space. Having one server named sandbox.* and the other ax102-01.* caused persistent confusion — config bugs, wrong healthUrls, agents misrouting.

S282.3 fixed two bugs:

  1. SELF_HOSTED_INFRA_JSON had healthUrl pointing to ax102-01.gograb.space instead of AX42's URL → dedup logic in HetznerInfraService.getSelfHostedEntries() silently dropped AX42 from the dashboard
  2. git.service.ts hardcoded fallback still pointed to sandbox.gograb.space → should be ax102-01.gograb.space (git repos moved to AX102 in S260)

S282.3 renamed all runtime code references (11 files) and cleaned up .env.local comments (documentation moved here).

2. Authoritative Server Reference

Server DNS IP API Key Roles
AX102-U ax102-01.gograb.space 178.63.165.83 hNwl3fka8Jc_xuUFvdvWin2IPMfI38sL5XuSpKQD1MI customer-sandbox (priority 1), git-operations
AX42-U ax42-01.gograb.space (was sandbox.gograb.space) 144.76.69.55 W8Wp0ltL8El_qRnXyoUoD0ffuSfZzmbn7Y7SVrx1k8s executive-sandbox, customer overflow (priority 2)

Server Details

Field AX42-U AX102-U
Hardware AX42-U #2938512 AX102-U #2946196
CPU Ryzen 7 PRO 8700GE (8C/16T) Ryzen 9 7950X3D (16C/32T)
RAM 128GB DDR5 128GB DDR5
Storage 2x512GB NVMe 2x1TB NVMe
Datacenter FSN1-DC10 FSN1-DC21
VM Capacity 30 60
SSH ssh [email protected] (key auth) ssh [email protected]

Environment Variable Mapping

Variable Value Why
SANDBOX_SERVERS_JSON Both servers with roles PlacementRouter multi-server config
SANDBOX_SERVER_URL https://ax102-01.gograb.space Primary sandbox server (legacy single-server fallback)
SANDBOX_API_KEY AX102-U's key Matches SANDBOX_SERVER_URL
SANDBOX_SERVER_LABEL AX102-U (Falkenstein DC21) Dashboard label for primary
SANDBOX_SERVER_HARDWARE Ryzen 9 7950X3D (16C/32T), 128GB DDR5, 2x1TB NVMe Dashboard hardware info
SELF_HOSTED_INFRA_JSON AX42-U at ax42-01.gograb.space Dashboard visibility for the second server
PLATFORM_GIT_SERVER_URL https://ax102-01.gograb.space Git repos live on AX102-U since S260
PLATFORM_GIT_API_KEY AX102-U's key Matches PLATFORM_GIT_SERVER_URL

3. Substitution Rules (for doc updates)

NOT every sandbox.gograb.space becomes ax42-01.gograb.space. Apply the correct rule:

Rule 1: AX42-U identity → ax42-01.gograb.space

Rule 2: Git server → ax102-01.gograb.space

Rule 3: Generic "primary sandbox URL" → ax102-01.gograb.space

Rule 4: Historical session reports → annotate, don't rewrite

4. Work Completed (S282.3)

Runtime files updated (11 files):

Cloudflare DNS — DONE

Railway — DONE

Server-Side (AX42-U) — DONE

5. Work Remaining

5A. Documentation (21 files, ~78 occurrences)

Done (S282.5). Sweep applied with substitution rules 1–4; canonical list for reference:

docs/english/sessions/S282/282.5-infrastructure-how-to-resolve-ax42-missing-from-dashboard-and-nam.md
docs/english/sessions/S261/261.5-bugfix-why-were-git-operations-still-routing-to-ax42-u-de.md
docs/english/sessions/S261/261.4-infrastructure-what-are-the-current-hetzner-server-roles-and-rout.md
docs/english/sessions/S260/260.3-architecture-how-should-the-sandboxplacementrouter-be-architect.md
docs/english/sessions/S252/S252-handoff.md
docs/english/sessions/S251/S251-progress-report.md
docs/english/sessions/S251/S251-handoff.md
docs/english/sessions/S245/245-daily-summary-2026-03-20.md
docs/english/Plans/superpowers/specs/archived/2026-03-23-hetzner-dual-server-infrastructure-design.md (12 occurrences)
docs/english/Plans/superpowers/specs/archived/2026-03-20-vitto-personal-whatsapp-channel-design.md (5 occurrences)
docs/english/Plans/superpowers/plans/archived/2026-03-20-platform-infrastructure-evolution.md (11 occurrences)
docs/english/Plans/already-done/february-26-plans/self-hosted-e2b.plan.md
docs/english/Plans/S260: Hetzner Multi-Server Infrastructure and SandboxPlacementRouter.md (8 occurrences)
docs/english/Plans/S259: VSCode Extension Native Git Operations Tab.md
docs/english/Plans/S254-handoff-vscode-clone-support.md (8 occurrences)
docs/english/Plans/S251: Git Operations Library and Hetzner Pipeline Fixes.md
docs/english/Deployment/sandbox-orchestrator-optional-railway-and-dashboard.md
docs/english/Deployment/hetzner-firecracker-operations.md
docs/english/sessions/S152/152-session-report.md
docs/english/sessions/S150/150-handoff-to-session-152.md
docs/english/sessions/S144/144-work-report.md

5B. MongoDB — MCP Decision Records

Canonical script (maintain in repo): tools/scripts/database/update-sandbox-dns-references.mongosh.js — updates sessions.description and sessions.context when context is a string; skips object-shaped context.

Run this script against both MongoDB clusters:

// ── Script: update-sandbox-dns-references.js ──
// Run with: mongosh "mongodb+srv://..." --file update-sandbox-dns-references.js

// ── ai-agents database ──
use('ai-agents');

const textFields = ['chosen', 'reasoning', 'question', 'task'];
const filter = {
  $or: textFields.map(f => ({ [f]: { $regex: /sandbox\.gograb\.space/ } }))
};

const count = db.decisions.countDocuments(filter);
print(`Found ${count} decisions with sandbox.gograb.space in ai-agents.decisions`);

if (count > 0) {
  const updates = {};
  textFields.forEach(f => {
    updates[f] = {
      $replaceAll: {
        input: `$${f}`,
        find: 'sandbox.gograb.space',
        replacement: 'ax42-01.gograb.space'
      }
    };
  });
  const result = db.decisions.updateMany(filter, [{ $set: updates }]);
  print(`Updated ${result.modifiedCount} documents in ai-agents.decisions`);
}

// ── ai-mcp-dev database ──
use('ai-mcp-dev');

// core_rules
const crFilter = { content: { $regex: /sandbox\.gograb\.space/ } };
const crCount = db.core_rules.countDocuments(crFilter);
print(`Found ${crCount} core_rules with sandbox.gograb.space`);
if (crCount > 0) {
  db.core_rules.updateMany(crFilter, [{
    $set: {
      content: {
        $replaceAll: { input: '$content', find: 'sandbox.gograb.space', replacement: 'ax42-01.gograb.space' }
      }
    }
  }]);
}

// learned_patterns
const lpFilter = {
  $or: [
    { pattern: { $regex: /sandbox\.gograb\.space/ } },
    { description: { $regex: /sandbox\.gograb\.space/ } }
  ]
};
const lpCount = db.learned_patterns.countDocuments(lpFilter);
print(`Found ${lpCount} learned_patterns with sandbox.gograb.space`);
if (lpCount > 0) {
  db.learned_patterns.updateMany(lpFilter, [{
    $set: {
      pattern: {
        $replaceAll: { input: '$pattern', find: 'sandbox.gograb.space', replacement: 'ax42-01.gograb.space' }
      },
      description: {
        $replaceAll: { input: '$description', find: 'sandbox.gograb.space', replacement: 'ax42-01.gograb.space' }
      }
    }
  }]);
}

// sessions
const sessFilter = {
  $or: [
    { description: { $regex: /sandbox\.gograb\.space/ } },
    { 'context': { $regex: /sandbox\.gograb\.space/ } }
  ]
};
const sessCount = db.sessions.countDocuments(sessFilter);
print(`Found ${sessCount} sessions with sandbox.gograb.space`);
if (sessCount > 0) {
  db.sessions.updateMany(sessFilter, [{
    $set: {
      description: {
        $replaceAll: { input: '$description', find: 'sandbox.gograb.space', replacement: 'ax42-01.gograb.space' }
      }
    }
  }]);
}

print('\n✅ MongoDB DNS rename complete');

6. Verification Checklist

# No remaining references in runtime code
grep -r "sandbox\.gograb\.space" --include="*.ts" --include="*.js" --include="*.sh" \
  --include="*.json" --include="*.conf" --exclude-dir=node_modules --exclude-dir=dist \
  --exclude-dir=.git --exclude-dir=sessions/manifests
# Expected: 0 results

# No remaining references in docs (after 5A is done)
grep -r "sandbox\.gograb\.space" docs/
# Expected: 0 results

# Both servers healthy
curl -sS https://ax102-01.gograb.space/health  # → {"status":"ok"}
curl -sS https://ax42-01.gograb.space/health   # → {"status":"ok"}

# Dashboard shows 2 servers
# Navigate to dev.gograb.space/infrastructure/hetzner (or localhost:6300)
# Both AX102-U and AX42-U should be RUNNING

# PlacementRouter fleet status (backend-ai-api logs on startup)
# Should log: "Fleet initialized: ax102-01 (customer-sandbox, git-operations) | ax42-01 (executive-sandbox, customer-sandbox)"

7. Server-Side Lessons Learned

nginx sites-enabled was a copy, not a symlink: On AX42-U, /etc/nginx/sites-enabled/sandbox-orchestrator was a regular file (copied Mar 21), not a symlink to sites-available/. Editing sites-available/ had no effect. Fixed by replacing with a proper symlink. Always verify ls -la /etc/nginx/sites-enabled/ before assuming symlinks.

certbot auto-install can fail with domain rename: certbot --nginx -d ax42-01.gograb.space issued the cert successfully but failed to install it ("Could not automatically find a matching server block"). This happens when certbot can't match the new domain to an existing server block. Fix: update nginx server_name first, then run certbot, or manually set ssl_certificate paths.

8. MCP Decision

Logged as dev_1774700586392_jb5wi96 — type: infrastructure, confidence: 0.95

Generated by Claude Code (Session 282.3) — March 28, 2026