haitham/claw3d
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add runtime seam, Hermes adapter support, and demo gateway mode (#89)

Browse Source 
* fix: include kanbanImmersive in immersiveOverlayActive calculation

When Kanban board is open, HUD elements (camera preset buttons, edit toolbar, overlays) should be suppressed. The kanbanImmersive flag was defined but not included in the immersiveOverlayActive condition, causing HUD elements to remain visible.

This fix adds kanbanImmersive to the immersiveOverlayActive calculation so HUD elements are properly hidden when the Kanban board is open.

Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>

* Fix: Hide mini status bar when Kanban immersive overlay is open

Wraps the bottom-left mini status bar (showing agent stats, vibe score, and
control hints) with !immersiveOverlayActive check to match the behavior of
other HUD elements like camera controls and toolbar.

This ensures the status bar is properly hidden when the Kanban board or any
other immersive overlay is active, maintaining a clean immersive experience.

Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>

* chore: drop unrelated package-lock line from branch

Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>

* universal-backend-plan

* backend-neutral runtime seam

* package.json update

* feat: add Hermes gateway adapter as alternative to OpenClaw

Adds a WebSocket adapter that lets Claw3D connect to a Hermes AI agent
runtime without any changes to the frontend. The adapter implements the
full Claw3D gateway protocol and bridges it to the Hermes HTTP API.

Changes:
- server/hermes-gateway-adapter.js: WebSocket bridge implementing the
  Claw3D gateway protocol against the Hermes HTTP API. Supports all
  core methods (agents, sessions, chat streaming, cron, config, files,
  approvals) and multi-agent orchestration via spawn_agent/delegate_task
  tools. Persists conversation history to ~/.hermes/clawd3d-history.json.
- scripts/clawd3d-start.sh: All-in-one startup script that launches
  Hermes, the adapter, and the Next.js dev server with auto port
  conflict resolution. Alias as `claw3d` for convenience.
- src/features/office/hooks/useCronAgents.ts: Hook that polls the
  gateway for cron-scheduled agents and surfaces them in the 3D office.
- package.json: adds `hermes-adapter` npm script
- .env.example: documents Hermes config vars
- docs/hermes-gateway.md: setup guide and protocol reference

Usage:
  npm run hermes-adapter   # start adapter (connect to http://localhost:8642)
  npm run dev              # start Claw3D, point browser at localhost:3000
  # or: bash scripts/clawd3d-start.sh  (starts everything automatically)

Both OpenClaw and Hermes are supported simultaneously — the gateway URL
in NEXT_PUBLIC_GATEWAY_URL determines which backend Claw3D connects to.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add read_agent_context tool for cross-agent coordination

Agents can now read each other's conversation history via the
read_agent_context tool, enabling the orchestrator to check what
a sub-agent has done before re-delegating work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: wire Hermes office UX and role-aware runtime updates

* feature update - demomode & hermes adapter

* fix lint blockers

* lintfix #2

* fix: stabilize retro office camera preset callbacks

* Initial plan

* fix: stabilize retro office overview preset hooks

Agent-Logs-Url: https://github.com/gsknnft/Claw3D/sessions/9cc71555-591e-44cf-aec4-25affbdcb405

Co-authored-by: gsknnft <123185582+gsknnft@users.noreply.github.com>

* feat: add truthful backend selection, Hermes adapter hardening, and demo gateway mode

* fix: address bugbot review and finalize backend selection

* fixed - onboarding and hermes calls

* office systems roadmap

* feat specs in docs

* specs ready

* feat: continue custom runtime seam and gateway alignment

* custom lane wired

* feat: add custom runtime provider path and office runtime alignment

* runtime fixes

* fix lukes findings

* fix lukes findings #2

* stable UI & connect screen page -> overlay

* better baseline for connection

* stable providers & ui rendering

* best launch yet

* nearly no gateway on reconnect

* auto reconnect last state

* fix: preserve selected runtime across reconnects

Keep backend selection aligned with the operator's chosen runtime instead of reviving a mismatched last-known-good adapter, and keep custom runtimes prompting for reconnect when Studio cannot auto-connect them.

Made-with: Cursor

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>
Co-authored-by: Elias Pfeffer <eliaspfeffer@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: iamlukethedev <lucas.guilherme@smartwayslfl.com>
This commit is contained in:
gsknnft
2026-04-02 16:27:24 -04:00
committed by GitHub
parent a997f13601
commit 083c146aac
77 changed files with 10045 additions and 399 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+82 -8
View File
@@ -29,17 +29,23 @@ You walk through your AI workplace.
## What Claw3D Is
OpenClaw is the intelligence and task-execution layer.
Claw3D is the visualization and interaction layer.
Today it can sit on top of:
- OpenClaw through the existing gateway flow
- Hermes through the bundled WebSocket adapter
- a direct HTTP `custom` runtime provider for orchestrator-backed stacks
- a built-in demo gateway for office exploration without a real agent framework
In practical terms, this app gives you:
- a live `/office` retro-office environment where agents appear as workers moving through a shared 3D world
- an `/office/builder` surface for editing and publishing office layouts
- a gateway-first architecture that keeps agent state in OpenClaw while Studio stores local UI preferences
- a gateway-first architecture that keeps runtime state in the connected backend while Studio stores local UI preferences
- a backend-neutral runtime seam inside Studio so additional providers can be integrated without rewriting the whole UI
This repository does not build or modify the OpenClaw runtime itself. It is the frontend and proxy layer that connects to an existing OpenClaw Gateway.
This repository does not build the upstream runtimes themselves. It is the frontend, Studio, and adapter/proxy layer that connects to a runtime speaking the Claw3D gateway protocol.
## Why It Exists
@@ -71,13 +77,16 @@ Requirements:
- Node.js 20+ recommended.
- npm 10+ recommended.
- A working OpenClaw installation with a reachable Gateway URL and token.
- One of:
- a working OpenClaw installation with a reachable Gateway URL and token
- Hermes with the bundled adapter
- the built-in demo gateway for local exploration
Prerequisite:
- Claw3D does not install, build, or run OpenClaw for you.
- Before starting Claw3D, make sure your OpenClaw gateway is already running and that you know the gateway URL and token you want Studio to use.
- This repository is the UI and Studio/proxy layer only.
- Claw3D does not install or build OpenClaw or Hermes for you.
- Before starting Claw3D against a real backend, make sure your chosen runtime is already running and that you know the gateway URL and token Studio should use.
- For a no-framework local office demo, run the bundled demo gateway instead.
- If you need a full cross-machine setup guide (OpenClaw + Tailscale + Claw3D), follow [`TUTORIAL.md`](TUTORIAL.md).
Run from source:
@@ -91,6 +100,65 @@ npm run dev
```
Then open `http://localhost:3000` and configure the gateway URL and token in Studio.
Studio now also persists the selected backend mode (`OpenClaw`, `Hermes`, `Demo`, or `Custom`) and
shows the active backend reported by the connected gateway.
### Custom runtime mode
If you are integrating an orchestrator-backed runtime through the `custom`
provider seam, start your runtime first, then start Claw3D:
```bash
npm run dev
```
Then open `http://localhost:3000`, choose `Custom backend`, and point the
upstream URL at your runtime boundary, for example:
```text
http://127.0.0.1:7770
```
Current `custom` runtime expectations:
- `GET /health`
- `GET /state`
- `GET /registry`
- `POST /v1/chat/completions`
The browser does not call that runtime directly. Claw3D proxies the
`custom` provider through its own same-origin route at
`/api/runtime/custom`, which avoids browser-side CORS problems and keeps
the provider transport separate from the OpenClaw/Hermes gateway path.
### Demo mode
If you only want to see the office and agent interactions without installing OpenClaw or Hermes:
```bash
npm run demo-gateway
npm run dev
```
Then connect Studio to:
```text
ws://localhost:18789
```
This starts a mock local gateway with demo agents, streaming chat, session previews, and office presence.
In the connect screen, choose `Demo backend`, then connect.
### Hermes adapter
If you want to use Hermes instead of OpenClaw:
```bash
npm run hermes-adapter
npm run dev
```
See [`docs/hermes-gateway.md`](docs/hermes-gateway.md) for setup details and current scope.
For a local gateway on the same machine, the usual upstream URL is:
@@ -98,6 +166,8 @@ For a local gateway on the same machine, the usual upstream URL is:
ws://localhost:18789
```
In the connect screen, choose `Hermes backend`, then connect.
## How It Connects
Claw3D uses two separate network hops:
@@ -168,6 +238,8 @@ See [`.env.example`](.env.example) for the full local development template.
## Scripts
- `npm run dev` starts the Studio dev server.
- `npm run hermes-adapter` starts the Hermes WebSocket adapter.
- `npm run demo-gateway` starts the built-in mock gateway for demo mode.
- `npm run build` builds the production Next.js app.
- `npm run start` starts the production server.
- `npm run lint` runs ESLint.
@@ -189,6 +261,7 @@ See [`.env.example`](.env.example) for the full local development template.
- [`ROADMAP.md`](ROADMAP.md): near-term priorities and contributor-friendly work areas.
- [`docs/pi-chat-streaming.md`](docs/pi-chat-streaming.md): gateway runtime streaming and transcript rendering.
- [`docs/permissions-sandboxing.md`](docs/permissions-sandboxing.md): Studio permissions and OpenClaw behavior.
- [`docs/hermes-gateway.md`](docs/hermes-gateway.md): Hermes adapter setup, capabilities, and current limitations.
## Current Limitations
@@ -202,6 +275,7 @@ If the UI loads but Connect fails, the problem is usually on the Studio -> Gatew
- Confirm the upstream URL and token in Studio settings.
- `EPROTO` or `wrong version number` usually means `wss://` was used against a non-TLS endpoint.
- `INVALID_REQUEST` errors mentioning `minProtocol` or `maxProtocol` usually mean the gateway is too old for Claw3D protocol v3. Upgrade OpenClaw, use the Hermes adapter, or run `npm run demo-gateway`.
- `401 Studio access token required` usually means `STUDIO_ACCESS_TOKEN` is enabled and the request is missing the expected `studio_access` cookie.
- Helpful proxy error codes include `studio.gateway_url_missing`, `studio.gateway_token_missing`, `studio.upstream_error`, and `studio.upstream_closed`.