Session 180: Vitto Chat — Persistence Fix + World-Class Improvements

Date: March 5-6, 2026 Branch: session-180 / session-181 Status: Phases 1-2 complete, Phases 3-4 planned

What This Session Accomplished (Plain Language)

Vitto's chat system had a critical problem: messages were not being saved to the server. When you sent a message on one device, it only existed on that device. If you opened a different browser, switched computers, or even refreshed quickly — your messages were gone.

On top of fixing that, we made the chat significantly faster, smarter, and more reliable. Here's the timeline of everything we did and why.

Timeline

Part 1: Emergency Fixes (Critical Bugs)

Fix 1: Messages Were Never Reaching the Server

What broke: Every message you sent was supposed to be saved to MongoDB (our database server) so it would be available on any device. Instead, messages were being silently queued in a local retry list and never sent.

Why it broke: A connectivity service (NetworkSyncStateService) starts in a "not connected" state and needs to be explicitly initialized to start checking if the server is available. In the Vitto app, nobody ever initialized it. So it permanently believed the server was unreachable, and every message went into a retry queue that also checked the same broken connectivity flag — creating a deadlock.

How we fixed it: Removed the dependency on the broken connectivity service. Now messages are sent directly to the server using the browser's built-in online/offline detection. If the server is actually down, the existing error handler catches it and queues for retry.

Impact: Messages now sync to MongoDB on every send. Conversations are visible across all devices.

Fix 2: Assistant Responses Were Never Saved

What broke: When Vitto responds to you, the response streams in word-by-word (like typing). During streaming, the message is marked as "in progress" and intentionally NOT saved to the server (because it's incomplete). But when streaming finished, nobody triggered the save.

How we fixed it: Added a trigger that detects when a streaming message is finalized and immediately syncs the complete response to the server.

Impact: Both your messages AND Vitto's responses now persist to the server.

Fix 3: Page Reload Could Lose Recent Messages

What broke: The local storage system (IndexedDB) has a 300-millisecond delay before writing changes to disk (to batch rapid updates). If you refreshed the page during that 300ms window, the latest message was lost.

How we fixed it: Added handlers that flush all pending writes immediately when you close the tab, refresh the page, or switch to another app on mobile.

Impact: No data loss on refresh, tab close, or mobile app switch.

Fix 4: Loading Sessions Could Scramble Data

What broke: When the chat page loads, it fires three requests simultaneously: (1) load the active session, (2) load session history, (3) reconcile any differences. These ran in parallel with no coordination. If request #2 finished before #1, it could delete sessions that #1 hadn't loaded yet.

How we fixed it: Sequenced the requests: active session loads first, then history merges (seeing the active session already in memory), then reconciliation runs on the final merged state.

Impact: Sessions load correctly every time, no phantom disappearances.

Fix 5: Expired Login Tokens Let Users See Empty Pages

What broke: When your login token expired, the app still let you access the dashboard because it only checked "does a token exist?" not "is the token still valid?" You'd see the full page layout with a "Sign in" button and no working features.

How we fixed it: The auth guard now decodes the JWT token, checks the expiration date, and redirects to the login page if expired. It also cleans up stale tokens from browser storage.

Impact: Expired sessions redirect to login immediately. No more ghost dashboard.

Part 2: Performance Improvements (Phase 1)

These changes make the chat feel instant. All were implemented in parallel with no dependencies on each other.

1.1: Faster Cost Tracking

Before: Every time a new message arrived, the system recalculated the total cost by looping through ALL messages in the session. With 200 messages, that's 200 calculations per new message.

After: Running totals are maintained incrementally. Each new message adds its cost to the running total in one step.

User impact: No lag when messages arrive in long conversations.

1.2: Smoother Session Search

Before: Every keystroke in the session search box triggered an immediate re-filter of all sessions. With 500+ sessions, this froze the UI briefly on each keystroke.

After: Search is debounced by 200 milliseconds — the filter only runs when you pause typing.

User impact: Smooth typing in the search box, no jank.

1.3: Stream Recovery

Before: If the network hiccupped during a streaming response, the entire stream died with no recovery. You'd lose the partial response.

After: The stream automatically retries up to 3 times with increasing delays (1 second, 2 seconds, 4 seconds). Only surfaces an error after all retries fail.

User impact: Transient network issues are handled transparently.

1.4: Thinking Indicator Timeout

Before: If the server never responded (crash, timeout, dropped connection), the "Thinking..." indicator would spin forever. You couldn't tell if Vitto was working or dead.

After: After 60 seconds, the indicator stops and a system message appears: "The response is taking longer than expected. You can try sending your message again."

User impact: Clear feedback when something goes wrong. No infinite spinners.

1.5: Smart Message Loading

Before: All 200+ messages in a session were rendered in the page at once, even though you can only see the latest few. This caused a 500+ millisecond lag on page load.

After: Only the last 30 messages render initially. When you scroll up, 20 more load automatically. A subtle "X earlier messages" indicator shows at the top. Your scroll position is preserved when older messages load.

User impact: Chat page loads instantly regardless of conversation length.

1.6: Batched Server Writes

Before: During a streaming response, 3-5 individual save requests fired in rapid succession (one per tool call + final response). Each was a separate round-trip to the server.

After: Messages are buffered for 500 milliseconds and sent in a single batch request.

User impact: Reduced server load and faster sync during active conversations.

Part 3: Intelligence Improvements (Phase 2)

These changes make Vitto smarter with every message.

2.1: Conversation Memory Compression

Before: When Vitto prepares to respond, it sends the last 10 raw messages as context (~5,000 tokens). This is expensive and misses important context from earlier in the conversation.

After: Vitto now uses a "Conversation Genome" — a compressed summary of the entire conversation (~500 tokens) that captures topics, decisions, facts, and user intent. This is prepended to the last 5 raw messages, giving Vitto 5-10x more context awareness in the same token budget.

User impact: Vitto remembers more from earlier in the conversation. Lower token costs.

2.2: Smart Follow-Up Suggestions

Before: After each response, Vitto showed generic follow-up buttons based on simple pattern matching: "Yes/No" after questions, "Thanks!" after completions. Always the same.

After: Two-tier system: (1) Pattern-based suggestions show instantly (no change in speed), (2) An AI-generated set of contextual suggestions replaces them ~300ms later. These are specific to the conversation context — e.g., "Deploy this?" after a build, "Check staging?" after a deploy.

User impact: More relevant, actionable follow-up options. Feels like Vitto understands what you're working on.

2.3: Remembering Your Model Preference

Before: If you switched from Claude Sonnet to Claude Opus, the selection was lost on page refresh.

After: Your preferred model is saved to browser storage and restored automatically on every page load.

User impact: Set your model once, it sticks.

2.4: Connection Status Awareness

Before: If the Vitto engine went down, the chat silently fell back to a basic AI mode with no tools, memory, or multi-agent capabilities. You had no way to know you were getting a degraded experience.

After: A "Direct mode" badge appears in the header when Vitto is unavailable, and a system message explains what features are limited. When Vitto recovers, the badge disappears automatically.

User impact: Transparent communication about what Vitto can do right now.

What's Next (Phases 3 and 4)

Phase 3: UX Polish (Planned)

• Cost badge formatting — Show "< 1 cent" instead of "$0.000000045"
• Accessibility labels — Screen reader support for all buttons and controls
• File upload validation — Reject files over 10MB with a clear error
• Streaming robustness — Fix edge case where finalized messages briefly show empty

Phase 4: World-Class Features (Planned)

• Vector search for memory — Sub-100ms semantic recall (currently 2-5 seconds)
• Conversation branching — Fork any conversation from any message to explore alternatives
• Cross-session intelligence — Vitto remembers what you discussed yesterday without being told
• Proactive messages — Vitto initiates conversation: "Your build failed — want me to check the logs?"

Files Modified

Critical Bug Fixes

Phase 1: Performance

Phase 2: Intelligence

MCP Intelligence Captured

Key Metrics