🏗️ Architecture: Custom Vision+JSON loop vs Anthropic Computer Use beta API f...

Date: 2026-02-24
Time: 14:40:34 UTC
Session: gspace1-138.9
Agent: backend
Confidence: 95%
Status: ✅ Documented

📋 Decision Context

Question:

Custom Vision+JSON loop vs Anthropic Computer Use beta API for computer-use-loop.service.ts

Task: S139 computer use upgrade to OpenClaw parity

🎯 Decision Made

Chosen:

Anthropic Computer Use beta API (betas: ['computer-use-2024-10-22'], model: claude-sonnet-4-5-20251022)

Options Considered:

• No alternatives documented

💭 Reasoning

The custom Vision+JSON approach was fragile: Claude returns JSON manually, parsing fails on markdown blocks, action types were custom-invented. The Anthropic Computer Use beta API gives Claude native computer tool support — it knows it's controlling a computer. No JSON parsing needed. Actions (screenshot, left_click, right_click, double_click, type, key, scroll, mouse_move, left_click_drag) are well-defined and atomic. This matches OpenClaw's direct machine control capability. Keep in backend-ai-api (shared infrastructure layer used by all agents via InfrastructureClientService HTTP bridge).

📁 Files Involved

• apis/backend-ai-api/src/app/infrastructure/services/computer-use-loop.service.ts
• libs/libs-ai/vitto-hub/src/lib/services/vitto-api.service.ts
• libs/libs-ai/vitto-hub/src/lib/components/computer-use-panel/computer-use-panel.component.ts

🔗 Related

• Decision ID: dev_1771944030970_wtyrzc2
• Session: 138
• Workspace: gspace1

Auto-generated by MCP Smart Consolidation