feat: add runtime seam, Hermes adapter support, and demo gateway mode (#89)
* fix: include kanbanImmersive in immersiveOverlayActive calculation When Kanban board is open, HUD elements (camera preset buttons, edit toolbar, overlays) should be suppressed. The kanbanImmersive flag was defined but not included in the immersiveOverlayActive condition, causing HUD elements to remain visible. This fix adds kanbanImmersive to the immersiveOverlayActive calculation so HUD elements are properly hidden when the Kanban board is open. Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com> * Fix: Hide mini status bar when Kanban immersive overlay is open Wraps the bottom-left mini status bar (showing agent stats, vibe score, and control hints) with !immersiveOverlayActive check to match the behavior of other HUD elements like camera controls and toolbar. This ensures the status bar is properly hidden when the Kanban board or any other immersive overlay is active, maintaining a clean immersive experience. Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com> * chore: drop unrelated package-lock line from branch Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com> * universal-backend-plan * backend-neutral runtime seam * package.json update * feat: add Hermes gateway adapter as alternative to OpenClaw Adds a WebSocket adapter that lets Claw3D connect to a Hermes AI agent runtime without any changes to the frontend. The adapter implements the full Claw3D gateway protocol and bridges it to the Hermes HTTP API. Changes: - server/hermes-gateway-adapter.js: WebSocket bridge implementing the Claw3D gateway protocol against the Hermes HTTP API. Supports all core methods (agents, sessions, chat streaming, cron, config, files, approvals) and multi-agent orchestration via spawn_agent/delegate_task tools. Persists conversation history to ~/.hermes/clawd3d-history.json. - scripts/clawd3d-start.sh: All-in-one startup script that launches Hermes, the adapter, and the Next.js dev server with auto port conflict resolution. Alias as `claw3d` for convenience. - src/features/office/hooks/useCronAgents.ts: Hook that polls the gateway for cron-scheduled agents and surfaces them in the 3D office. - package.json: adds `hermes-adapter` npm script - .env.example: documents Hermes config vars - docs/hermes-gateway.md: setup guide and protocol reference Usage: npm run hermes-adapter # start adapter (connect to http://localhost:8642) npm run dev # start Claw3D, point browser at localhost:3000 # or: bash scripts/clawd3d-start.sh (starts everything automatically) Both OpenClaw and Hermes are supported simultaneously — the gateway URL in NEXT_PUBLIC_GATEWAY_URL determines which backend Claw3D connects to. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add read_agent_context tool for cross-agent coordination Agents can now read each other's conversation history via the read_agent_context tool, enabling the orchestrator to check what a sub-agent has done before re-delegating work. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: wire Hermes office UX and role-aware runtime updates * feature update - demomode & hermes adapter * fix lint blockers * lintfix #2 * fix: stabilize retro office camera preset callbacks * Initial plan * fix: stabilize retro office overview preset hooks Agent-Logs-Url: https://github.com/gsknnft/Claw3D/sessions/9cc71555-591e-44cf-aec4-25affbdcb405 Co-authored-by: gsknnft <123185582+gsknnft@users.noreply.github.com> * feat: add truthful backend selection, Hermes adapter hardening, and demo gateway mode * fix: address bugbot review and finalize backend selection * fixed - onboarding and hermes calls * office systems roadmap * feat specs in docs * specs ready * feat: continue custom runtime seam and gateway alignment * custom lane wired * feat: add custom runtime provider path and office runtime alignment * runtime fixes * fix lukes findings * fix lukes findings #2 * stable UI & connect screen page -> overlay * better baseline for connection * stable providers & ui rendering * best launch yet * nearly no gateway on reconnect * auto reconnect last state * fix: preserve selected runtime across reconnects Keep backend selection aligned with the operator's chosen runtime instead of reviving a mismatched last-known-good adapter, and keep custom runtimes prompting for reconnect when Studio cannot auto-connect them. Made-with: Cursor --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com> Co-authored-by: Elias Pfeffer <eliaspfeffer@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: iamlukethedev <lucas.guilherme@smartwayslfl.com>
This commit is contained in:
gsknnft
@@ -0,0 +1,316 @@
|
||||
# Custom Provider Reference
|
||||
|
||||
> Reference implementation guide for plugging a non-OpenClaw, non-Hermes runtime into Claw3D through the upstream-safe `custom` provider seam.
|
||||
|
||||
## Goal
|
||||
|
||||
Show how a custom orchestration stack should plug into Claw3D without requiring:
|
||||
|
||||
- a named built-in provider
|
||||
- OpenClaw emulation
|
||||
- Hermes adapter semantics
|
||||
|
||||
The shape should be:
|
||||
|
||||
- Claw3D sees `custom`
|
||||
- the external runtime stays implementation-specific
|
||||
- Claw3D core remains generic
|
||||
|
||||
## Current Implementation Notes
|
||||
|
||||
The current `custom` branch path is deliberately conservative.
|
||||
|
||||
What exists today:
|
||||
|
||||
- provider selection and metadata flow through the Studio runtime seam
|
||||
- same-origin runtime proxying through `/api/runtime/custom`
|
||||
- health, state, and registry probing
|
||||
- direct chat via `/v1/chat/completions`
|
||||
- office/bootstrap/chat/model loading routed through the provider layer
|
||||
|
||||
What still needs to mature:
|
||||
|
||||
- true provider-native streaming
|
||||
- stronger multi-session persistence semantics
|
||||
- integration tests against a live custom runtime
|
||||
- richer office presentation of runtime metadata, lanes, and model identity
|
||||
|
||||
## Position
|
||||
|
||||
A custom runtime should sit at the Claw3D boundary as an orchestrator-backed service.
|
||||
|
||||
That means:
|
||||
|
||||
- Claw3D should talk to one stable runtime boundary
|
||||
- that boundary may route work to internal worker processes, models, or lanes
|
||||
- those internal details should remain mostly hidden behind the provider
|
||||
|
||||
This keeps the integration:
|
||||
|
||||
- stable
|
||||
- upstream-safe
|
||||
- reusable for multiple custom stacks later
|
||||
|
||||
## Why The Custom Provider Exists
|
||||
|
||||
Not every useful runtime should need a named upstream provider branch.
|
||||
|
||||
Some stacks will be:
|
||||
|
||||
- internal
|
||||
- experimental
|
||||
- organization-specific
|
||||
- built around custom orchestration
|
||||
|
||||
The `custom` provider exists so those stacks can integrate cleanly without forcing Claw3D core to absorb stack-specific assumptions.
|
||||
|
||||
## Recommended Boundary
|
||||
|
||||
Claw3D should integrate with the custom runtime's orchestrator or gateway layer, not with its individual workers.
|
||||
|
||||
Recommended public boundary:
|
||||
|
||||
- `POST /v1/chat/completions`
|
||||
- `POST /v1/completions`
|
||||
- `POST /v1/contracted-completions`
|
||||
- `GET /health`
|
||||
- `GET /state`
|
||||
- `GET /registry`
|
||||
|
||||
These do not need to be mandatory for every implementation, but they are a strong reference shape because they provide:
|
||||
|
||||
- chat entry points
|
||||
- reachability
|
||||
- runtime summary
|
||||
- model or role registry visibility
|
||||
|
||||
## Runtime Shape
|
||||
|
||||
Recommended mapping:
|
||||
|
||||
```text
|
||||
Claw3D
|
||||
-> custom provider
|
||||
-> custom orchestrator / gateway
|
||||
-> internal routing
|
||||
-> workers / roles / models / tools
|
||||
```
|
||||
|
||||
This means the custom provider should not need to know:
|
||||
|
||||
- worker ports
|
||||
- shard startup semantics
|
||||
- internal runtime policy
|
||||
- proprietary state heuristics
|
||||
|
||||
It only needs:
|
||||
|
||||
- request/response surfaces
|
||||
- route or session metadata
|
||||
- health/state/registry metadata
|
||||
|
||||
## Provider Identity
|
||||
|
||||
Upstream-facing identity should remain:
|
||||
|
||||
- `providerId: "custom"`
|
||||
|
||||
Implementation-specific identity should appear as metadata, not provider class naming.
|
||||
|
||||
Suggested metadata:
|
||||
|
||||
```ts
|
||||
type CustomRuntimeMetadata = {
|
||||
id: "custom";
|
||||
label: "Custom Runtime";
|
||||
runtimeName?: string | null;
|
||||
vendor?: string | null;
|
||||
runtimeVersion?: string | null;
|
||||
routeProfile?: string | null;
|
||||
};
|
||||
```
|
||||
|
||||
This gives the UI enough identity without forcing upstream Claw3D to branch on every runtime brand.
|
||||
|
||||
## Capability Profile
|
||||
|
||||
Initial `custom` provider capability claims should be conservative and honest.
|
||||
|
||||
Likely V1 support for many custom runtimes:
|
||||
|
||||
- `agents`
|
||||
- `sessions`
|
||||
- `chat`
|
||||
- `streaming`
|
||||
- `agent_roles`
|
||||
- `config`
|
||||
|
||||
Possible later support depending on implementation:
|
||||
|
||||
- `files`
|
||||
- `approvals`
|
||||
- `skills`
|
||||
- `cron`
|
||||
- `session_settings`
|
||||
|
||||
Important rule:
|
||||
|
||||
Do not claim support just because the backend can theoretically do something. Claim support only where the provider exposes a stable Claw3D-facing behavior.
|
||||
|
||||
## Agent Mapping
|
||||
|
||||
Claw3D agents should not be modeled as one fixed backend process each unless the runtime truly behaves that way.
|
||||
|
||||
Instead, a custom runtime may map agents to:
|
||||
|
||||
- roles
|
||||
- lanes
|
||||
- strategies
|
||||
- departments
|
||||
- routed execution identities
|
||||
|
||||
Example office-facing identities:
|
||||
|
||||
- `Main`
|
||||
- `Assistant`
|
||||
- `Coder`
|
||||
- `Reviewer`
|
||||
- `Professor`
|
||||
|
||||
The exact labels are implementation-specific.
|
||||
|
||||
The important point is that the provider should expose office-meaningful identities rather than leaking raw backend topology.
|
||||
|
||||
## Session Mapping
|
||||
|
||||
Claw3D sessions should map to runtime conversations or execution threads, not to whichever backend storage structure happens to exist internally.
|
||||
|
||||
Recommended session metadata:
|
||||
|
||||
- `sessionKey`
|
||||
- conversation or thread id
|
||||
- active role
|
||||
- lane
|
||||
- requested model
|
||||
- resolved model
|
||||
- request id
|
||||
|
||||
The provider should normalize these into a stable session model no matter what the backend calls them internally.
|
||||
|
||||
## Event Mapping
|
||||
|
||||
The provider should translate runtime activity into the same normalized Claw3D runtime events used elsewhere.
|
||||
|
||||
Recommended mappings:
|
||||
|
||||
- runtime agent change -> `presence.changed`
|
||||
- conversation/session update -> `session.activity`
|
||||
- streaming token output -> `chat.delta`
|
||||
- final assistant turn -> `chat.final`
|
||||
- routed or backend failure -> `chat.error`
|
||||
- request lifecycle -> `run.lifecycle`
|
||||
- tool or workflow progress -> `tool.progress`
|
||||
|
||||
Implementation-specific metadata is allowed, but only as additive metadata.
|
||||
|
||||
## Route Metadata
|
||||
|
||||
Many custom runtimes will have useful execution metadata.
|
||||
|
||||
The provider may expose optional metadata such as:
|
||||
|
||||
- selected role
|
||||
- candidate roles
|
||||
- lane
|
||||
- routing reason
|
||||
- registry profile
|
||||
- resolved model id
|
||||
- worker or runtime health summary
|
||||
|
||||
This metadata should be visible in diagnostics or advanced panels, not required for basic user flow.
|
||||
|
||||
## State And Health Surfaces
|
||||
|
||||
The custom provider should prefer orchestrator-level health/state queries first.
|
||||
|
||||
Recommended usage:
|
||||
|
||||
- `/health`
|
||||
- basic runtime reachability
|
||||
- `/state`
|
||||
- route profile, active roles, worker/runtime summary
|
||||
- `/registry`
|
||||
- active model, role, or profile catalog
|
||||
|
||||
Worker-level status endpoints should stay behind the provider unless the UI needs a diagnostic drill-down.
|
||||
|
||||
## Relationship To Agent State
|
||||
|
||||
The custom provider is the right place to map richer internal runtime signals into Claw3D's public office state model.
|
||||
|
||||
Public office state should remain simple:
|
||||
|
||||
- `focused`
|
||||
- `working`
|
||||
- `blocked`
|
||||
- `overloaded`
|
||||
- `degraded`
|
||||
|
||||
Internally, a custom stack may derive those from richer signals such as:
|
||||
|
||||
- route stress
|
||||
- worker saturation
|
||||
- runtime policy
|
||||
- confidence or control signals
|
||||
- proprietary orchestration heuristics
|
||||
|
||||
That should remain implementation-private.
|
||||
|
||||
Claw3D only needs the resulting office-safe state and maybe a public reason label.
|
||||
|
||||
## Relationship To Office Systems
|
||||
|
||||
The custom provider should support office systems without Claw3D needing to know the backend's internals.
|
||||
|
||||
Examples:
|
||||
|
||||
- bulletin board gets agent/session/task context
|
||||
- whiteboard gets planning-session context
|
||||
- meetings get coordination state
|
||||
- QA gets review or readiness metadata
|
||||
|
||||
Again, the provider should expose only what the office needs.
|
||||
|
||||
## Suggested V1 Scope
|
||||
|
||||
Recommended first `custom` provider scope:
|
||||
|
||||
1. Reach the custom orchestrator over `/v1/chat/completions`.
|
||||
2. Pull metadata from `/health`, `/state`, and `/registry`.
|
||||
3. Map runtime roles or routes into Claw3D agent identities.
|
||||
4. Surface streaming and final turns.
|
||||
5. Expose route metadata in a diagnostics-friendly way.
|
||||
6. Map simple public office states from observable runtime conditions.
|
||||
|
||||
This is enough to make a custom runtime useful in Claw3D without overexposing internals.
|
||||
|
||||
## Suggested V2 Scope
|
||||
|
||||
Once the V1 provider is stable, add:
|
||||
|
||||
- richer session persistence
|
||||
- role-aware office teams
|
||||
- custom runtime diagnostics panel
|
||||
- office state enrichment from internal stack signals
|
||||
- hooks into bulletin board, whiteboard, QA, and meeting systems
|
||||
|
||||
## Explicit Non-Goals
|
||||
|
||||
This reference should not require:
|
||||
|
||||
- runtime-specific branches throughout Claw3D core
|
||||
- OpenClaw compatibility shims
|
||||
- routing logic duplicated in the frontend
|
||||
- direct worker orchestration in the browser
|
||||
|
||||
The provider should remain the containment layer.
|
||||
@@ -0,0 +1,300 @@
|
||||
# Custom Runtime Provider Spec
|
||||
|
||||
> Generic extension seam for non-OpenClaw, non-Hermes stacks.
|
||||
|
||||
## Goal
|
||||
|
||||
Define a clean `custom` runtime provider class for Claw3D.
|
||||
|
||||
This provider should let external orchestration stacks integrate with Claw3D through a stable seam without requiring:
|
||||
|
||||
- OpenClaw emulation
|
||||
- Hermes-specific semantics
|
||||
- named first-class core support for every stack
|
||||
|
||||
The idea is:
|
||||
|
||||
- upstream concept: `custom` provider
|
||||
- downstream implementations provide their own runtime behavior against that seam
|
||||
|
||||
## Current Branch Status
|
||||
|
||||
On `dev/vera_lane`, the `custom` provider is no longer just a design
|
||||
placeholder.
|
||||
|
||||
Current implemented behavior:
|
||||
|
||||
- `custom` is a first-class provider ID in the runtime seam
|
||||
- Studio persists the selected backend mode as `custom`
|
||||
- the provider exposes runtime metadata such as `runtimeName`, `vendor`,
|
||||
`runtimeVersion`, and `routeProfile` when available
|
||||
- Claw3D probes `GET /health`, `GET /state`, and `GET /registry`
|
||||
- chat uses a direct HTTP path to `POST /v1/chat/completions`
|
||||
- browser traffic is proxied through Claw3D's same-origin
|
||||
`/api/runtime/custom` route instead of calling the runtime directly
|
||||
|
||||
Still intentionally missing in this branch:
|
||||
|
||||
- normalized streaming event support
|
||||
- richer session persistence beyond the synthetic provider session layer
|
||||
- direct approvals/files/cron surfaces
|
||||
- process auto-launch from Studio
|
||||
|
||||
## Why This Matters
|
||||
|
||||
Not every useful runtime should have to become a named built-in provider in upstream Claw3D.
|
||||
|
||||
A clean custom provider seam gives:
|
||||
|
||||
- extensibility
|
||||
- lower upstream friction
|
||||
- room for proprietary or stack-specific orchestration
|
||||
- a path for advanced internal systems without polluting core abstractions
|
||||
|
||||
This is especially important when a stack is:
|
||||
|
||||
- internal
|
||||
- organization-specific
|
||||
- experimental
|
||||
- orchestration-heavy
|
||||
|
||||
## Product Position
|
||||
|
||||
The `custom` provider should be treated as:
|
||||
|
||||
- a first-class extension seam
|
||||
- not a hack
|
||||
- not a vendor-specific side path
|
||||
|
||||
This is the upstream-safe abstraction.
|
||||
|
||||
## Relationship To Existing Provider Work
|
||||
|
||||
Claw3D’s runtime abstraction already points toward multiple providers:
|
||||
|
||||
- `openclaw`
|
||||
- `hermes`
|
||||
- future providers
|
||||
|
||||
The `custom` provider should sit alongside those as a generic lane for:
|
||||
|
||||
- external orchestrators
|
||||
- private agent stacks
|
||||
- organization-specific routing systems
|
||||
|
||||
## Core Principle
|
||||
|
||||
Do not leak implementation-specific internal logic into the generic provider contract.
|
||||
|
||||
The provider should expose:
|
||||
|
||||
- common runtime behaviors
|
||||
- provider capabilities
|
||||
- normalized events
|
||||
- optional metadata
|
||||
|
||||
It should not require upstream Claw3D to know:
|
||||
|
||||
- runtime-specific routing internals
|
||||
- proprietary state logic
|
||||
- private planning models
|
||||
- stack-specific heuristics
|
||||
|
||||
## Provider Identity
|
||||
|
||||
Suggested provider IDs:
|
||||
|
||||
- `openclaw`
|
||||
- `hermes`
|
||||
- `custom`
|
||||
|
||||
Then allow custom metadata such as:
|
||||
|
||||
```ts
|
||||
type CustomRuntimeDescriptor = {
|
||||
providerId: "custom";
|
||||
runtimeName: string;
|
||||
runtimeVersion?: string | null;
|
||||
vendor?: string | null;
|
||||
capabilities?: string[];
|
||||
};
|
||||
```
|
||||
|
||||
This gives Claw3D enough identity for UI without hardcoding a brand into the provider class.
|
||||
|
||||
## Reference Implementation Strategy
|
||||
|
||||
The `custom` provider should exist as a generic extension seam.
|
||||
|
||||
That means:
|
||||
|
||||
- upstream Claw3D gets `custom`
|
||||
- downstream stacks supply their own behavior
|
||||
- others can later implement their own custom providers against the same seam
|
||||
|
||||
That is much easier to justify upstream than:
|
||||
|
||||
- adding a deeply stack-specific built-in provider before the generic extension seam exists
|
||||
|
||||
## Suggested Contract Shape
|
||||
|
||||
This should align with the existing runtime abstraction, but allow custom metadata.
|
||||
|
||||
Example:
|
||||
|
||||
```ts
|
||||
type RuntimeProviderId = "openclaw" | "hermes" | "custom";
|
||||
|
||||
type RuntimeProviderMetadata = {
|
||||
id: RuntimeProviderId;
|
||||
label: string;
|
||||
runtimeName?: string | null;
|
||||
runtimeVersion?: string | null;
|
||||
vendor?: string | null;
|
||||
};
|
||||
```
|
||||
|
||||
The `custom` provider should still implement the same base runtime methods:
|
||||
|
||||
- list agents
|
||||
- list sessions
|
||||
- send chat
|
||||
- abort run
|
||||
- wait for run
|
||||
- stream normalized events
|
||||
- expose capabilities honestly
|
||||
|
||||
## Capability Philosophy
|
||||
|
||||
The custom provider should be capability-driven, not assumption-driven.
|
||||
|
||||
That means if a custom stack supports:
|
||||
|
||||
- roles
|
||||
- approvals
|
||||
- files
|
||||
- cron
|
||||
- whiteboard integration
|
||||
- meeting signals
|
||||
|
||||
it should declare those clearly.
|
||||
|
||||
If it does not, the UI should degrade honestly.
|
||||
|
||||
## Event Model
|
||||
|
||||
The custom provider should emit the same normalized event categories as other providers.
|
||||
|
||||
Examples:
|
||||
|
||||
- `presence.changed`
|
||||
- `session.activity`
|
||||
- `chat.delta`
|
||||
- `chat.final`
|
||||
- `chat.error`
|
||||
- `run.lifecycle`
|
||||
- `tool.progress`
|
||||
|
||||
This keeps Claw3D stable even if the custom stack has richer private event semantics internally.
|
||||
|
||||
## Custom Metadata Surface
|
||||
|
||||
The custom provider may optionally expose richer metadata for display.
|
||||
|
||||
Examples:
|
||||
|
||||
- routed lane
|
||||
- active model id
|
||||
- strategy label
|
||||
- execution mode
|
||||
- custom stack status
|
||||
|
||||
Important:
|
||||
|
||||
This metadata should be additive and optional.
|
||||
|
||||
It should not change the core runtime contract.
|
||||
|
||||
## Reference Implementation Model
|
||||
|
||||
A custom provider can reasonably map:
|
||||
|
||||
- agents -> routed roles / lanes
|
||||
- sessions -> runtime conversations
|
||||
- streaming -> orchestrator or gateway output streams
|
||||
- provider metadata -> runtime name, lane, model, or route state
|
||||
|
||||
The public upstream concept remains `custom`.
|
||||
|
||||
Implementation-specific mapping stays in the runtime layer.
|
||||
|
||||
## Relationship To Agent State Model
|
||||
|
||||
The custom provider is the right place for richer internal state to enter Claw3D.
|
||||
|
||||
For example:
|
||||
|
||||
- a custom runtime computes deeper control or state signals
|
||||
- the provider maps them into public office states
|
||||
|
||||
This keeps:
|
||||
|
||||
- internal intelligence private
|
||||
- public office state understandable
|
||||
|
||||
## Relationship To Office Systems
|
||||
|
||||
The custom provider should be able to support office systems without Claw3D needing to know the backend’s internals.
|
||||
|
||||
Examples:
|
||||
|
||||
- bulletin board gets agent/session/task context
|
||||
- whiteboard gets planning-session context
|
||||
- meetings get coordination state
|
||||
- QA gets review or readiness metadata
|
||||
|
||||
Again, the custom provider should expose only what the office needs.
|
||||
|
||||
## V1 Scope
|
||||
|
||||
Recommended V1 scope:
|
||||
|
||||
- define `custom` provider identity
|
||||
- allow custom provider metadata
|
||||
- ensure capability-driven UI behavior
|
||||
- make no runtime-specific assumptions in core Claw3D
|
||||
|
||||
Actual runtime adapters can be implemented separately against that seam.
|
||||
|
||||
## Out of Scope For V1
|
||||
|
||||
- embedding proprietary internal orchestration logic in Claw3D core
|
||||
- hardcoding any one runtime as upstream architecture
|
||||
- requiring all custom providers to support advanced signals
|
||||
|
||||
The point is generic extensibility first.
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
Recommended order:
|
||||
|
||||
1. Add `custom` as a first-class runtime provider identity.
|
||||
2. Add a metadata surface for runtime name/vendor/version.
|
||||
3. Ensure the provider factory supports `custom`.
|
||||
4. Keep the runtime event and capability model normalized.
|
||||
5. Implement Vera as the first reference adapter outside the generic contract.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
This spec is successful if:
|
||||
|
||||
- upstream Claw3D gains a clean extension seam
|
||||
- Vera can integrate without bloating core architecture
|
||||
- future stacks can follow the same pattern
|
||||
- the office systems continue to work against normalized runtime behavior
|
||||
|
||||
## Summary
|
||||
|
||||
The `custom` runtime provider is the right upstream abstraction for stack-specific orchestrators.
|
||||
|
||||
It gives Claw3D extensibility, gives Vera a clean path in, and avoids hardwiring a personal stack directly into the core product model.
|
||||
@@ -0,0 +1,264 @@
|
||||
# Universal Backend Plan
|
||||
|
||||
> Backend-neutral Claw3D integration plan for OpenClaw, Hermes, Vera, and other runtimes.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Do not treat PR #70 as the long-term integration architecture.
|
||||
|
||||
It is useful as a short-term compatibility shim and a source of a few good UX changes, but it does not make Claw3D backend-neutral. It keeps Claw3D OpenClaw-shaped and makes Hermes imitate OpenClaw.
|
||||
|
||||
That matters because:
|
||||
|
||||
- Hermes already has real control surfaces: ACP and an OpenAI-compatible API server.
|
||||
- Vera already has a real orchestrator/gateway shape.
|
||||
- Every future backend would otherwise need to keep emulating the OpenClaw gateway protocol.
|
||||
|
||||
The better path is:
|
||||
|
||||
1. Keep OpenClaw support intact.
|
||||
2. Extract a backend-neutral runtime adapter inside Claw3D.
|
||||
3. Add Hermes and Vera providers against their native surfaces where possible.
|
||||
4. Cherry-pick the high-value UI pieces from PR #70 into that new architecture.
|
||||
|
||||
## What To Reuse From PR #70
|
||||
|
||||
These are worth keeping:
|
||||
|
||||
- Multi-agent UX concepts.
|
||||
- `read_agent_context` as a coordination primitive.
|
||||
- Agent `role` flowing into the 3D office nameplate.
|
||||
- Click-to-chat behavior.
|
||||
- Live speech bubble rendering for streaming text.
|
||||
- Hermes-specific env var documentation.
|
||||
|
||||
These are not the right long-term seam:
|
||||
|
||||
- A full OpenClaw-protocol emulator as the primary Hermes integration.
|
||||
- Fake-success implementations for `config.*` and approvals.
|
||||
- Synthesizing runtime freshness from `Date.now()` instead of real event/message timestamps.
|
||||
|
||||
## Target Architecture
|
||||
|
||||
Claw3D should stop treating the browser gateway client as the backend abstraction.
|
||||
|
||||
Instead, Studio should expose a backend-neutral runtime service with provider adapters:
|
||||
|
||||
```text
|
||||
Browser UI
|
||||
-> Studio runtime API
|
||||
-> OpenClaw provider
|
||||
-> Hermes provider
|
||||
-> Vera provider
|
||||
```
|
||||
|
||||
The browser can still use WebSocket streaming from Studio, but the messages should be Claw3D-native runtime events rather than implicitly OpenClaw events.
|
||||
|
||||
## Core Adapter Contract
|
||||
|
||||
Suggested TypeScript shape:
|
||||
|
||||
```ts
|
||||
export type RuntimeCapability =
|
||||
| "agents"
|
||||
| "sessions"
|
||||
| "chat"
|
||||
| "streaming"
|
||||
| "agent_roles"
|
||||
| "files"
|
||||
| "skills"
|
||||
| "cron"
|
||||
| "approvals"
|
||||
| "config"
|
||||
| "session_settings";
|
||||
|
||||
export type RuntimeEvent =
|
||||
| { type: "presence.changed"; agents: RuntimeAgentSummary[] }
|
||||
| { type: "session.activity"; sessionKey: string; agentId: string; at: number }
|
||||
| { type: "chat.delta"; runId: string; sessionKey: string; text: string; at: number }
|
||||
| { type: "chat.final"; runId: string; sessionKey: string; text: string; at: number }
|
||||
| { type: "chat.error"; runId: string; sessionKey: string; message: string; at: number }
|
||||
| { type: "run.lifecycle"; runId: string; sessionKey: string; phase: "start" | "end" | "error"; at: number }
|
||||
| { type: "tool.progress"; runId: string; sessionKey: string; label: string; at: number };
|
||||
|
||||
export interface RuntimeProvider {
|
||||
readonly id: string;
|
||||
readonly label: string;
|
||||
getCapabilities(): Promise<Set<RuntimeCapability>>;
|
||||
connect(): Promise<void>;
|
||||
disconnect(): Promise<void>;
|
||||
subscribe(listener: (event: RuntimeEvent) => void): () => void;
|
||||
listAgents(): Promise<RuntimeAgentSummary[]>;
|
||||
listSessions(input?: { agentId?: string }): Promise<RuntimeSessionSummary[]>;
|
||||
getSessionPreview(keys: string[]): Promise<RuntimeSessionPreview[]>;
|
||||
sendChat(input: { sessionKey: string; message: string; agentId?: string }): Promise<{ runId: string }>;
|
||||
abortRun(input: { runId?: string; sessionKey?: string }): Promise<void>;
|
||||
waitForRun(input: { runId: string; timeoutMs?: number }): Promise<"running" | "done">;
|
||||
}
|
||||
```
|
||||
|
||||
Optional features such as config editing, approvals, files, skills, and cron should sit behind capability checks instead of being assumed to exist.
|
||||
|
||||
## Capability Matrix
|
||||
|
||||
Initial expected support:
|
||||
|
||||
| Capability | OpenClaw | Hermes | Vera |
|
||||
|---|---|---|---|
|
||||
| Agents | Native | Native | Provider-defined |
|
||||
| Sessions | Native | Native | Provider-defined |
|
||||
| Chat send/abort/wait | Native | Native | Native via orchestrator |
|
||||
| Streaming | Native | Native | Native |
|
||||
| Agent roles | Native-ish | Native | Native |
|
||||
| Files | Native | Partial | Optional |
|
||||
| Skills | Native | Native | Optional |
|
||||
| Cron | Native | Native | Optional |
|
||||
| Approvals | Native | Partial | Optional |
|
||||
| Config mutation | Native | Limited | Limited |
|
||||
|
||||
Important rule:
|
||||
|
||||
If a provider does not support a surface, Claw3D should disable or hide the UI for it. It should not fake a successful write.
|
||||
|
||||
## Provider Strategy
|
||||
|
||||
### OpenClaw Provider
|
||||
|
||||
Use the existing gateway client as the first provider implementation.
|
||||
|
||||
This keeps current behavior working while the rest of the app migrates to the adapter contract.
|
||||
|
||||
### Hermes Provider
|
||||
|
||||
Preferred order:
|
||||
|
||||
1. ACP for session-aware agent orchestration.
|
||||
2. Hermes API server for OpenAI-compatible chat and streaming.
|
||||
3. OpenClaw-protocol shim only as a temporary bridge.
|
||||
|
||||
Rationale:
|
||||
|
||||
- ACP is a better semantic fit for sessions, cancellation, fork/resume, approvals, and editor-style state.
|
||||
- The Hermes API server is already stable and useful for chat, tool calling, and cron-backed service behavior.
|
||||
- The OpenClaw shim should be treated as transitional compatibility, not the permanent contract.
|
||||
|
||||
### Vera Provider
|
||||
|
||||
Target the Vera orchestrator, not individual `vera-torch` workers.
|
||||
|
||||
Use:
|
||||
|
||||
- `POST /v1/chat/completions`
|
||||
- `POST /v1/completions`
|
||||
- `POST /v1/contracted-completions`
|
||||
- `GET /health`
|
||||
- `GET /state`
|
||||
- `GET /registry`
|
||||
|
||||
The Vera provider should map Claw3D agent identities to routed roles or lanes rather than pretending Vera is an OpenClaw gateway.
|
||||
|
||||
## Event Model
|
||||
|
||||
Current Claw3D expects OpenClaw-flavored `chat`, `agent`, and `presence` events.
|
||||
|
||||
That is too narrow for universal providers. Studio should normalize provider-native updates into a Claw3D event model with explicit semantics:
|
||||
|
||||
- `presence.changed`
|
||||
- `session.activity`
|
||||
- `chat.delta`
|
||||
- `chat.final`
|
||||
- `chat.error`
|
||||
- `run.lifecycle`
|
||||
- `tool.progress`
|
||||
|
||||
Then the browser UI can consume one stable event shape no matter what backend is in use.
|
||||
|
||||
## High-Value PR Split
|
||||
|
||||
Recommended implementation order:
|
||||
|
||||
### PR 1: Runtime Abstraction
|
||||
|
||||
Scope:
|
||||
|
||||
- Introduce the provider interface.
|
||||
- Wrap current OpenClaw behavior in an `openclaw` provider.
|
||||
- Move capability checks into the UI state layer.
|
||||
- Add a Studio-level runtime event normalization layer.
|
||||
|
||||
This is the most important PR.
|
||||
|
||||
### PR 2: Safe UX Cherry-Picks From PR #70
|
||||
|
||||
Scope:
|
||||
|
||||
- Agent `role` in store and office UI.
|
||||
- Click-to-chat.
|
||||
- Streaming speech bubbles.
|
||||
|
||||
These are good product improvements and do not require committing to the Hermes shim architecture.
|
||||
|
||||
### PR 3: Hermes Native Provider
|
||||
|
||||
Scope:
|
||||
|
||||
- Add a `hermes` provider using ACP where possible.
|
||||
- Use Hermes API server for chat/streaming surfaces.
|
||||
- Expose capabilities honestly.
|
||||
- Persist and surface real timestamps from Hermes session/message state.
|
||||
|
||||
Keep the shim optional for compatibility, not required.
|
||||
|
||||
### PR 4: Vera Provider
|
||||
|
||||
Scope:
|
||||
|
||||
- Add a `vera` provider against the Vera orchestrator.
|
||||
- Map Claw3D agents to Vera roles or lanes.
|
||||
- Surface orchestrator state and routed worker identity.
|
||||
|
||||
### PR 5: Optional Compatibility Layer Cleanup
|
||||
|
||||
Scope:
|
||||
|
||||
- Retire or reduce the Hermes OpenClaw shim.
|
||||
- Convert shim-only routes into provider-native routes where possible.
|
||||
|
||||
## Near-Term Guidance For Luke
|
||||
|
||||
If Luke wants "drop-in Hermes support right now", PR #70 is directionally useful.
|
||||
|
||||
If Luke wants "Claw3D should support any backend cleanly", PR #70 should not be the mainline architecture.
|
||||
|
||||
Best compromise:
|
||||
|
||||
- Do not merge PR #70 as the final backend architecture.
|
||||
- Split out the UI improvements and any safe Hermes-specific pieces.
|
||||
- Open a new architecture PR for the runtime provider seam.
|
||||
- Rebase Hermes integration on top of that seam.
|
||||
|
||||
## Why This Also Helps Vera
|
||||
|
||||
This path avoids making Vera imitate OpenClaw.
|
||||
|
||||
Instead, Vera can appear as:
|
||||
|
||||
- a routed multi-role intelligence backend,
|
||||
- with Claw3D visualizing agents, runs, status, and streamed text,
|
||||
- while preserving Vera-specific routing, lane, and model identity.
|
||||
|
||||
That gives Claw3D a broader identity:
|
||||
|
||||
- similar to the OpenClaw ecosystem,
|
||||
- but not subordinate to OpenClaw's protocol and assumptions.
|
||||
|
||||
## Proposed First Deliverable
|
||||
|
||||
The first concrete deliverable should be a new PR that does only this:
|
||||
|
||||
- add the provider interface,
|
||||
- wrap existing OpenClaw integration behind it,
|
||||
- add capability flags,
|
||||
- make the UI stop assuming config/approval/file support from every backend.
|
||||
|
||||
That PR creates the seam both Hermes and Vera need.
|
||||
Reference in New Issue
Block a user