API Dev Tools — Implementation Plan (Session 229)

Branch: session-229-230 (concurrent with session-230)
Scope: Backend instrumentation first, then frontend. One unified route with filtering.
Revision: v3 — incorporates v2 re-analysis (forRoot pattern, SSE exclusion, LoggingInterceptor recommendation, WebSocket out-of-scope).

1. Executive Summary

Goal: Build API Dev Tools in gograb-dev so developers can see real-time activity across all 4 APIs, filter by endpoint/user/role/event type, and generate reports.

Order: Backend instrumentation → Frontend. No shortcuts.

User/role: backend-api is the canonical source for users and roles. All APIs log userId (from JWT when present). Frontend fetches user list (with roles) from backend-api and correlates with activity for role-based filtering.

2. Architecture Decisions

2.1 One Route vs Per-API Routes

Decision: One unified route with filter-by-API.

• /api-tools — Overview (all 4 APIs health + summary)
• /api-tools/activity — Activity feed (filter by API, endpoint, user, role, event type, record type)
• /api-tools/reports — Report generation (export filtered data)

2.2 User/Role Filtering

Decision: backend-api is the canonical source.

• backend-api: Owns users, roles, auth. JWT payload includes sub (userId), platformRole, roles.
• All APIs: Log userId from req.user?.userId or req.user?.sub when authenticated. Log role from req.user?.platformRole or req.user?.roles?.[0] when available.
• Frontend: Fetches user list (with roles) from backend-api. For "filter by role": get userIds with that role, filter activity where userId in that set. No need for every API to call backend-api on each request.

Assumption verified: JWT strategy maps sub → userId in req.user (unified-auth.guard.ts → createUserCaller()).

2.3 Shared Backend Library

Decision: Add request activity instrumentation to @libs-api/backend-core.

• All 4 APIs already use @libs-api/backend-core (CORS, validation, Redis, etc.).
• New: RequestActivityInterceptor + RequestActivityStore + RequestActivityModule + DevActivityController.
• DRY: One implementation, imported by all APIs.

2.4 Frontend Library Category

Decision: @libs-structural/api-tools — NOT @libs-api.

• libs/libs-api/ is exclusively for NestJS backend libraries (backend-core, auth-core, etc.).
• libs/libs-structural/ contains route-level Angular libraries consumed by apps (deployment-system, flow-engine-lib, vitto-studio, etc.).
• API Dev Tools is an Angular route-level feature → structural.

3. Phase 1: Backend Instrumentation

3.1 Add to @libs-api/backend-core

Record type (4-tier classification):

Alternative (3-tier): success (2xx–3xx), client_error (4xx), server_error (5xx). Never conflate redirects with errors.

excludePaths: Config to skip logging certain paths. Default: ['/dev/', '/health', '/system/']. Prevents polling noise (frontend polls /dev/activity and /health every 5–10s) from flooding the ring buffer. With 4 APIs × 2 endpoints × 2 polls/min ≈ 96 requests/min of noise; without exclusion, 2000-record buffer fills in ~20 minutes.

SSE long-lived connections: The codebase has 15+ SSE endpoints (Vitto engine, module generation, MCP gateway, deployment, etc.). SSE connections stay open for minutes or hours. Without exclusion, the interceptor would record durationMs: 300000+ and a single statusCode when the connection closes — producing misleading "ultra-slow request" entries. Add SSE paths to excludePaths (e.g., /engine/process, /module-generation/generate, /mcp/, /deployment/status) or detect content-type: text/event-stream and skip recording. Option 1 (path exclusion) is simplest.

Do NOT log: Request/response bodies, query params (could contain tokens). Only: method, path (as-is for dev tools), statusCode, userId, role, duration, timestamp.

3.2 Activity Endpoint Contract

GET /api/dev/activity      (backend-api)
GET /ai-api/dev/activity  (backend-ai-api)
GET /docs-api/dev/activity (backend-docs-api)
GET /vitto-api/dev/activity (backend-vitto-api)

Query params:

Response:

{
  "api": "backend-api",
  "records": [
    {
      "requestId": "req_xxx",
      "method": "GET",
      "path": "/api/users/me",
      "statusCode": 200,
      "userId": "usr_abc",
      "role": "developer",
      "durationMs": 45,
      "timestamp": "2026-03-17T16:00:00.000Z",
      "recordType": "success"
    }
  ],
  "total": 1
}

3.3 LoggingInterceptor Overlap

Problem: Existing LoggingInterceptor already extracts method, url, statusCode, durationMs, requestId. Duplicating in RequestActivityInterceptor is wasteful.

Solution (choose one):

1. Extend LoggingInterceptor: Add optional callback/hook that receives the extracted data; RequestActivityStore subscribes. No second extraction.
2. Compose: RequestActivityInterceptor delegates to LoggingInterceptor for extraction, then pushes to store.
3. Refactor LoggingInterceptor: Extract a shared RequestMetrics type and extractRequestMetrics(req, res, startTime) utility; both interceptors use it.

Recommended: Option 3. Cleanest approach: doesn't modify LoggingInterceptor's behavior, gives both interceptors a stable contract, and avoids coupling.

3.4 Integration Per API

Each API (backend-api, backend-ai-api, backend-docs-api, backend-vitto-api):

1. Import RequestActivityModule.forRoot() from @libs-api/backend-core — dynamic module pattern (same as RedisModule.forRoot(), AuthCoreModule.forRoot()).
2. Pass API-specific config so the controller knows which API it's running in and can populate the response api field:

RequestActivityModule.forRoot({
  apiName: 'backend-api',     // → populates response `api` field
  bufferSize: 2000,           // optional override
  excludePaths: ['/dev/', '/health', '/system/', '/engine/process', '/module-generation/generate', '/mcp/', '/deployment/status'],  // optional; includes SSE paths
})

3. Register in app module.
4. Ensure RequestActivityInterceptor runs after auth guards (NestJS lifecycle: Guards → Interceptors → Pipes → Handler). Use APP_INTERCEPTOR.

Path consideration: Controller @Controller('dev') + @Get('activity') under backend-api (prefix api) → /api/dev/activity. Verified: NestJS global prefix applies to all controllers.

3.5 Auth for Dev Activity Endpoint

Decision: Require authentication. Use @Roles('developer', 'admin', 'system') decorator on DevActivityController (idiomatic NestJS; avoid manual in-handler role check like system/explorer).

Defense-in-depth: DevActivityGuard checks NODE_ENV !== 'production' || process.env.ENABLE_DEV_ACTIVITY === 'true'. In production without the flag, return 404. Even if ENABLE_DEV_ACTIVITY leaks to prod, auth still protects.

4. Phase 2: Frontend

4.1 Create @libs-structural/api-tools Library

Category: libs-structural (route-level Angular). NOT libs-api (NestJS backend).

4.2 Routes in gograb-dev

Relationship to /infrastructure: The existing /infrastructure route shows agent/session/pipeline stats. API Tools shows request-level API activity. Complementary, not overlapping. Nav: Infrastructure under "System", API Tools as a new section.

4.3 Data Flow

• Health: Poll /api/health, /ai-api/health, /docs-api/health, /vitto-api/health.
• Activity: Poll or fetch from each API's /dev/activity. Per-API error isolation: each API call wrapped in catchError returning of(null) or { api, error, records: [] }. Overview shows "backend-vitto-api: unreachable" while others display data.
• Users (for filter dropdown): Fetch from backend-api. Add GET /api/dev/users (dev-only, admin role) if none exists.
• Roles: From backend-api (user response or roles endpoint).
• excludePaths client-side: Hide /dev/ and /health from the activity feed by default (filter out records matching excludePaths). Reduces noise when viewing "real" activity.

4.4 Environment: vittoApiUrl

Gap: environment.ts has backendApiUrl, aiApiUrl, docsApiUrl but not vittoApiUrl. proxy.conf.json already routes /vitto-api → localhost:3003.

Fix: Add vittoApiUrl: VITTO_API_URLS.localProxy to environment.ts. Import VITTO_API_URLS from @libs-foundational/api-config.

4.5 Real-Time

• Polling: Every 5–10 seconds for activity when on activity tab. Configurable.
• WebSocket: Future enhancement. Not in initial scope.

5. Implementation Order

Phase 1 (Backend)

Phase 2 (Frontend)

6. Risks and Mitigations

7. Out of Scope (Initial)

• WebSocket streaming (polling for activity)
• WebSocket message activity capture — NestJS HTTP interceptors don't capture WebSocket messages. The codebase has 8 WebSocket gateways across all 4 APIs. Excluding from v1 by design; future implementers should not assume it was forgotten.
• Persistent storage (MongoDB) for activity
• Request/response body capture
• Production deployment of dev activity (guard disables by default)

8. Assumptions (Verified)

9. Verification Checklist

• All 4 APIs expose GET /{prefix}/dev/activity with auth
• Each API uses RequestActivityModule.forRoot({ apiName: '...' }) — response api field correct
• Activity records include userId, role when authenticated
• Record type: success, redirect, client_error, server_error (4-tier)
• excludePaths excludes /dev/, /health, /system/, and SSE paths from logging
• Filter by userId, role, endpoint, recordType works
• gograb-dev shows unified view with API filter
• Per-API error isolation: one down API doesn't fail entire overview
• vittoApiUrl in dev environment
• Report export (CSV/JSON) works with current filters
• No regression: existing health/infrastructure routes still work

10. Revision History