System-optimization: Session 180 Phase 1: How to make Vitto chat performance feel...

Documentation for Sessions

📝 System-optimization: Session 180 Phase 1: How to make Vitto chat performance feel...

Date: 2026-03-06
Time: 15:57:42 UTC
Session: gspace1-180.8
Agent: angular
Confidence: 95%
Status: ✅ Documented

📋 Decision Context

Question:

Session 180 Phase 1: How to make Vitto chat performance feel instant across 6 dimensions (cost tracking, search, SSE, timeout, rendering, batching)?

Task: Phase 1: Performance and reliability improvements for Vitto chat

🎯 Decision Made

Chosen:

Six parallel optimizations: (1) O(1) incremental cost aggregation replacing O(n) reduce, (2) 200ms debounced session search, (3) SSE retry with 3-attempt exponential backoff, (4) 60s typing indicator timeout with system message, (5) reverse-load architecture showing last 30 messages with scroll-up pagination, (6) 500ms message write batching reducing rapid HTTP calls to 1.

Options Considered:

  1. CDK virtual scroll for message list (rejected — variable height causes flicker)
  2. Reverse-load architecture with scroll pagination (chosen for 1.5)
  3. Web Worker for session search filtering (overkill — debounce sufficient)
  4. requestAnimationFrame batching for cost (overkill — O(1) signal update sufficient)

💭 Reasoning

Each optimization targets a different bottleneck: (1) Cost: sessionTokens/sessionCost computed signals ran reduce() over ALL messages on every update — O(n) per message arrival. At 200 messages that's 200 iterations per keystroke. Replaced with running total signals updated additively in O(1). (2) Search: Each keystroke in session search triggered synchronous re-filtering of all sessions. With 500+ sessions, this blocks the UI thread ~50ms per key. 200ms debounce via RxJS Subject eliminates jank. (3) SSE: A single network hiccup killed the entire stream with no recovery. Users lost partial responses. 3 retries with 1s/2s/4s backoff handles transient failures. HTTP 4xx/5xx errors are NOT retried (server rejection). (4) Timeout: If API hangs, typing indicator spins forever. 60s timeout auto-stops and injects system message so user knows to retry. (5) Rendering: CDK virtual scroll was considered but rejected — variable-height chat messages cause flicker/jump with virtual scroll. Reverse-load (render last 30, load 20 more on scroll-up, preserve scroll position) is simpler, works perfectly with variable heights, and gives same performance win. (6) Batching: During streaming, 3-5 individual appendMessages() calls fire in rapid succession. A 500ms buffer per session batches them into 1 HTTP write.

📁 Files Involved

🔗 Related

Auto-generated by MCP Smart Consolidation