feat: add runtime seam, Hermes adapter support, and demo gateway mode (#89)

* fix: include kanbanImmersive in immersiveOverlayActive calculation

When Kanban board is open, HUD elements (camera preset buttons, edit toolbar, overlays) should be suppressed. The kanbanImmersive flag was defined but not included in the immersiveOverlayActive condition, causing HUD elements to remain visible.

This fix adds kanbanImmersive to the immersiveOverlayActive calculation so HUD elements are properly hidden when the Kanban board is open.

Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>

* Fix: Hide mini status bar when Kanban immersive overlay is open

Wraps the bottom-left mini status bar (showing agent stats, vibe score, and
control hints) with !immersiveOverlayActive check to match the behavior of
other HUD elements like camera controls and toolbar.

This ensures the status bar is properly hidden when the Kanban board or any
other immersive overlay is active, maintaining a clean immersive experience.

Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>

* chore: drop unrelated package-lock line from branch

Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>

* universal-backend-plan

* backend-neutral runtime seam

* package.json update

* feat: add Hermes gateway adapter as alternative to OpenClaw

Adds a WebSocket adapter that lets Claw3D connect to a Hermes AI agent
runtime without any changes to the frontend. The adapter implements the
full Claw3D gateway protocol and bridges it to the Hermes HTTP API.

Changes:
- server/hermes-gateway-adapter.js: WebSocket bridge implementing the
  Claw3D gateway protocol against the Hermes HTTP API. Supports all
  core methods (agents, sessions, chat streaming, cron, config, files,
  approvals) and multi-agent orchestration via spawn_agent/delegate_task
  tools. Persists conversation history to ~/.hermes/clawd3d-history.json.
- scripts/clawd3d-start.sh: All-in-one startup script that launches
  Hermes, the adapter, and the Next.js dev server with auto port
  conflict resolution. Alias as `claw3d` for convenience.
- src/features/office/hooks/useCronAgents.ts: Hook that polls the
  gateway for cron-scheduled agents and surfaces them in the 3D office.
- package.json: adds `hermes-adapter` npm script
- .env.example: documents Hermes config vars
- docs/hermes-gateway.md: setup guide and protocol reference

Usage:
  npm run hermes-adapter   # start adapter (connect to http://localhost:8642)
  npm run dev              # start Claw3D, point browser at localhost:3000
  # or: bash scripts/clawd3d-start.sh  (starts everything automatically)

Both OpenClaw and Hermes are supported simultaneously — the gateway URL
in NEXT_PUBLIC_GATEWAY_URL determines which backend Claw3D connects to.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add read_agent_context tool for cross-agent coordination

Agents can now read each other's conversation history via the
read_agent_context tool, enabling the orchestrator to check what
a sub-agent has done before re-delegating work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: wire Hermes office UX and role-aware runtime updates

* feature update - demomode & hermes adapter

* fix lint blockers

* lintfix #2

* fix: stabilize retro office camera preset callbacks

* Initial plan

* fix: stabilize retro office overview preset hooks

Agent-Logs-Url: https://github.com/gsknnft/Claw3D/sessions/9cc71555-591e-44cf-aec4-25affbdcb405

Co-authored-by: gsknnft <123185582+gsknnft@users.noreply.github.com>

* feat: add truthful backend selection, Hermes adapter hardening, and demo gateway mode

* fix: address bugbot review and finalize backend selection

* fixed - onboarding and hermes calls

* office systems roadmap

* feat specs in docs

* specs ready

* feat: continue custom runtime seam and gateway alignment

* custom lane wired

* feat: add custom runtime provider path and office runtime alignment

* runtime fixes

* fix lukes findings

* fix lukes findings #2

* stable UI & connect screen page -> overlay

* better baseline for connection

* stable providers & ui rendering

* best launch yet

* nearly no gateway on reconnect

* auto reconnect last state

* fix: preserve selected runtime across reconnects

Keep backend selection aligned with the operator's chosen runtime instead of reviving a mismatched last-known-good adapter, and keep custom runtimes prompting for reconnect when Studio cannot auto-connect them.

Made-with: Cursor

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Luke The Dev <iamlukethedev@users.noreply.github.com>
Co-authored-by: Elias Pfeffer <eliaspfeffer@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: iamlukethedev <lucas.guilherme@smartwayslfl.com>
This commit is contained in:
gsknnft
2026-04-02 16:27:24 -04:00
committed by GitHub
parent a997f13601
commit 083c146aac
77 changed files with 10045 additions and 399 deletions
+513
View File
@@ -0,0 +1,513 @@
"use strict";
const http = require("http");
const { randomUUID } = require("crypto");
const { WebSocketServer } = require("ws");
const ADAPTER_PORT = parseInt(process.env.DEMO_ADAPTER_PORT || "18789", 10);
const MAIN_KEY = "main";
const MODELS = [{ id: "demo/mock-office", name: "Mock Office", provider: "demo" }];
const agents = new Map([
[
"demo-orchestrator",
{
id: "demo-orchestrator",
name: "Avery",
role: "Orchestrator",
workspace: "/demo/orchestrator",
},
],
[
"demo-researcher",
{
id: "demo-researcher",
name: "Mika",
role: "Research",
workspace: "/demo/research",
},
],
[
"demo-builder",
{
id: "demo-builder",
name: "Rune",
role: "Builder",
workspace: "/demo/builder",
},
],
]);
const files = new Map();
const sessionSettings = new Map();
const conversationHistory = new Map();
const activeRuns = new Map();
const activeSendEventFns = new Set();
function randomId() {
return randomUUID().replace(/-/g, "");
}
function sessionKeyFor(agentId) {
return `agent:${agentId}:${MAIN_KEY}`;
}
function getHistory(sessionKey) {
if (!conversationHistory.has(sessionKey)) {
conversationHistory.set(sessionKey, []);
}
return conversationHistory.get(sessionKey);
}
function clearHistory(sessionKey) {
conversationHistory.delete(sessionKey);
}
function resOk(id, payload) {
return { type: "res", id, ok: true, payload: payload ?? {} };
}
function resErr(id, code, message) {
return { type: "res", id, ok: false, error: { code, message } };
}
function broadcastEvent(frame) {
for (const send of activeSendEventFns) {
try {
send(frame);
} catch {}
}
}
function agentListPayload() {
return [...agents.values()].map((agent) => ({
id: agent.id,
name: agent.name,
workspace: agent.workspace,
identity: { name: agent.name, emoji: "🤖" },
role: agent.role,
}));
}
function buildDemoReply(agent, message) {
const normalized = message.trim();
const opening =
agent.role === "Orchestrator"
? `${agent.name} here. Demo office is live and the team is synced.`
: `${agent.name} reporting in from the ${agent.role.toLowerCase()} desk.`;
const action =
agent.role === "Research"
? "I would break this down into sources, constraints, and next questions."
: agent.role === "Builder"
? "I would turn that into concrete implementation steps and validation."
: "I can coordinate the team, route work, and summarize progress.";
return `${opening} You said: "${normalized}". ${action}`;
}
async function handleMethod(method, params, id, sendEvent) {
const p = params || {};
switch (method) {
case "agents.list":
return resOk(id, { defaultId: "demo-orchestrator", mainKey: MAIN_KEY, agents: agentListPayload() });
case "agents.create": {
const name = typeof p.name === "string" && p.name.trim() ? p.name.trim() : "Demo Agent";
const role = typeof p.role === "string" ? p.role.trim() : "";
const slug = name.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, "") || "demo-agent";
const agentId = `${slug}-${randomId().slice(0, 6)}`;
agents.set(agentId, {
id: agentId,
name,
role,
workspace: `/demo/${slug}`,
});
broadcastEvent({
type: "event",
event: "presence",
payload: { sessions: { recent: [], byAgent: [] } },
});
return resOk(id, { agentId, name, workspace: `/demo/${slug}` });
}
case "agents.update": {
const agentId = typeof p.agentId === "string" ? p.agentId.trim() : "";
const agent = agents.get(agentId);
if (!agent) return resErr(id, "not_found", `Agent ${agentId} not found`);
if (typeof p.name === "string" && p.name.trim()) agent.name = p.name.trim();
if (typeof p.role === "string") agent.role = p.role.trim();
return resOk(id, { ok: true, removedBindings: 0 });
}
case "agents.delete": {
const agentId = typeof p.agentId === "string" ? p.agentId.trim() : "";
if (agentId && agents.has(agentId) && agentId !== "demo-orchestrator") {
agents.delete(agentId);
clearHistory(sessionKeyFor(agentId));
}
return resOk(id, { ok: true, removedBindings: 0 });
}
case "agents.files.get": {
const key = `${p.agentId || "demo-orchestrator"}/${p.name || ""}`;
const content = files.get(key);
return resOk(id, { file: content !== undefined ? { content } : { missing: true } });
}
case "agents.files.set": {
const key = `${p.agentId || "demo-orchestrator"}/${p.name || ""}`;
files.set(key, typeof p.content === "string" ? p.content : "");
return resOk(id, {});
}
case "config.get":
return resOk(id, {
config: { gateway: { reload: { mode: "hot" } } },
hash: "demo-gateway",
exists: true,
path: "/demo/config.json",
});
case "config.patch":
case "config.set":
return resOk(id, { hash: "demo-gateway" });
case "exec.approvals.get":
return resOk(id, {
path: "",
exists: true,
hash: "demo-approvals",
file: { version: 1, defaults: { security: "full", ask: "off", autoAllowSkills: true }, agents: {} },
});
case "exec.approvals.set":
return resOk(id, { hash: "demo-approvals" });
case "exec.approval.resolve":
return resOk(id, { ok: true });
case "models.list":
return resOk(id, { models: MODELS });
case "skills.status":
return resOk(id, { skills: [] });
case "cron.list":
return resOk(id, { jobs: [] });
case "cron.add":
case "cron.run":
case "cron.remove":
return resErr(id, "unsupported_method", `Demo runtime does not support ${method}.`);
case "sessions.list": {
const sessions = [...agents.values()].map((agent) => {
const sessionKey = sessionKeyFor(agent.id);
const history = getHistory(sessionKey);
const settings = sessionSettings.get(sessionKey) || {};
return {
key: sessionKey,
agentId: agent.id,
updatedAt: history.length > 0 ? Date.now() : null,
displayName: "Main",
origin: { label: agent.name, provider: "demo" },
model: settings.model || MODELS[0].id,
modelProvider: "demo",
};
});
return resOk(id, { sessions });
}
case "sessions.preview": {
const keys = Array.isArray(p.keys) ? p.keys : [];
const limit = typeof p.limit === "number" ? p.limit : 8;
const maxChars = typeof p.maxChars === "number" ? p.maxChars : 240;
const previews = keys.map((key) => {
const history = getHistory(key);
if (history.length === 0) return { key, status: "empty", items: [] };
const items = history.slice(-limit).map((msg) => ({
role: msg.role === "assistant" ? "assistant" : "user",
text: String(msg.content || "").slice(0, maxChars),
timestamp: Date.now(),
}));
return { key, status: "ok", items };
});
return resOk(id, { ts: Date.now(), previews });
}
case "sessions.patch": {
const key = typeof p.key === "string" ? p.key : sessionKeyFor("demo-orchestrator");
const current = sessionSettings.get(key) || {};
const next = { ...current };
if (p.model !== undefined) next.model = p.model;
if (p.thinkingLevel !== undefined) next.thinkingLevel = p.thinkingLevel;
sessionSettings.set(key, next);
return resOk(id, {
ok: true,
key,
entry: { thinkingLevel: next.thinkingLevel },
resolved: { model: next.model || MODELS[0].id, modelProvider: "demo" },
});
}
case "sessions.reset": {
const key = typeof p.key === "string" ? p.key : sessionKeyFor("demo-orchestrator");
clearHistory(key);
return resOk(id, { ok: true });
}
case "chat.send": {
const sessionKey = typeof p.sessionKey === "string" ? p.sessionKey : sessionKeyFor("demo-orchestrator");
const agentId = sessionKey.startsWith("agent:") ? sessionKey.split(":")[1] : "demo-orchestrator";
const agent = agents.get(agentId) || agents.get("demo-orchestrator");
const message = typeof p.message === "string" ? p.message.trim() : String(p.message || "").trim();
const runId = typeof p.idempotencyKey === "string" && p.idempotencyKey ? p.idempotencyKey : randomId();
if (!message) return resOk(id, { status: "no-op", runId });
const reply = buildDemoReply(agent, message);
let aborted = false;
activeRuns.set(runId, {
runId,
sessionKey,
agentId,
abort() {
aborted = true;
},
});
setImmediate(async () => {
let seq = 0;
const emitChat = (state, extra) => {
sendEvent({
type: "event",
event: "chat",
seq: seq++,
payload: { runId, sessionKey, state, ...extra },
});
};
try {
const words = reply.split(" ");
let partial = "";
for (const word of words) {
if (aborted) break;
partial = partial ? `${partial} ${word}` : word;
emitChat("delta", { message: { role: "assistant", content: partial } });
await new Promise((resolve) => setTimeout(resolve, 45));
}
if (aborted) {
emitChat("aborted", {});
return;
}
const history = getHistory(sessionKey);
history.push({ role: "user", content: message });
history.push({ role: "assistant", content: reply });
emitChat("final", { stopReason: "end_turn", message: { role: "assistant", content: reply } });
sendEvent({
type: "event",
event: "presence",
seq: seq++,
payload: {
sessions: {
recent: [{ key: sessionKey, updatedAt: Date.now() }],
byAgent: [{ agentId, recent: [{ key: sessionKey, updatedAt: Date.now() }] }],
},
},
});
} finally {
activeRuns.delete(runId);
}
});
return resOk(id, { status: "started", runId });
}
case "chat.abort": {
const runId = typeof p.runId === "string" ? p.runId.trim() : "";
const sessionKey = typeof p.sessionKey === "string" ? p.sessionKey.trim() : "";
let aborted = 0;
if (runId) {
const handle = activeRuns.get(runId);
if (handle) {
handle.abort();
activeRuns.delete(runId);
aborted += 1;
}
} else if (sessionKey) {
for (const [activeRunId, handle] of activeRuns.entries()) {
if (handle.sessionKey !== sessionKey) continue;
handle.abort();
activeRuns.delete(activeRunId);
aborted += 1;
}
}
return resOk(id, { ok: true, aborted });
}
case "chat.history": {
const sessionKey = typeof p.sessionKey === "string" ? p.sessionKey : sessionKeyFor("demo-orchestrator");
return resOk(id, { sessionKey, messages: getHistory(sessionKey) });
}
case "agent.wait": {
const runId = typeof p.runId === "string" ? p.runId : "";
const timeoutMs = typeof p.timeoutMs === "number" ? p.timeoutMs : 30000;
const start = Date.now();
while (activeRuns.has(runId) && Date.now() - start < timeoutMs) {
await new Promise((resolve) => setTimeout(resolve, 50));
}
return resOk(id, { status: activeRuns.has(runId) ? "running" : "done" });
}
case "status": {
const recent = [...agents.keys()].flatMap((agentId) => {
const key = sessionKeyFor(agentId);
const history = getHistory(key);
return history.length > 0 ? [{ key, updatedAt: Date.now() }] : [];
});
return resOk(id, {
sessions: {
recent,
byAgent: [...agents.keys()].map((agentId) => ({
agentId,
recent: recent.filter((entry) => entry.key.includes(`:${agentId}:`)),
})),
},
});
}
case "wake":
return resOk(id, { ok: true });
default:
return resOk(id, {});
}
}
function startAdapter() {
const httpServer = http.createServer((req, res) => {
res.writeHead(200, { "Content-Type": "text/plain" });
res.end("Claw3D Demo Gateway Adapter\n");
});
const wss = new WebSocketServer({ server: httpServer });
wss.on("connection", (ws) => {
let connected = false;
let globalSeq = 0;
const send = (frame) => {
if (ws.readyState !== ws.OPEN) return;
ws.send(JSON.stringify(frame));
};
const sendEventFn = (frame) => {
if (frame.type === "event" && typeof frame.seq !== "number") {
frame.seq = globalSeq++;
}
send(frame);
};
activeSendEventFns.add(sendEventFn);
send({ type: "event", event: "connect.challenge", payload: { nonce: randomId() } });
ws.on("message", async (raw) => {
let frame;
try {
frame = JSON.parse(raw.toString("utf8"));
} catch {
return;
}
if (!frame || typeof frame !== "object" || frame.type !== "req") return;
const { id, method, params } = frame;
if (typeof id !== "string" || typeof method !== "string") return;
if (method === "connect") {
connected = true;
send({
type: "res",
id,
ok: true,
payload: {
type: "hello-ok",
protocol: 3,
adapterType: "demo",
features: {
methods: [
"agents.list",
"agents.create",
"agents.delete",
"agents.update",
"sessions.list",
"sessions.preview",
"sessions.patch",
"sessions.reset",
"chat.send",
"chat.abort",
"chat.history",
"agent.wait",
"status",
"config.get",
"config.set",
"config.patch",
"agents.files.get",
"agents.files.set",
"exec.approvals.get",
"exec.approvals.set",
"exec.approval.resolve",
"wake",
"skills.status",
"models.list",
"cron.list",
],
events: ["chat", "presence", "heartbeat"],
},
snapshot: {
health: {
agents: [...agents.values()].map((agent) => ({
agentId: agent.id,
name: agent.name,
isDefault: agent.id === "demo-orchestrator",
})),
defaultAgentId: "demo-orchestrator",
},
sessionDefaults: { mainKey: MAIN_KEY },
},
auth: { role: "operator", scopes: ["operator.admin"] },
policy: { tickIntervalMs: 30000 },
},
});
return;
}
if (!connected) {
send(resErr(id, "not_connected", "Send connect first."));
return;
}
try {
send(await handleMethod(method, params, id, sendEventFn));
} catch (error) {
send(resErr(id, "internal_error", error instanceof Error ? error.message : "Internal error"));
}
});
ws.on("close", () => activeSendEventFns.delete(sendEventFn));
ws.on("error", () => activeSendEventFns.delete(sendEventFn));
});
httpServer.listen(ADAPTER_PORT, "127.0.0.1", () => {
console.log(`[demo-gateway] Listening on ws://localhost:${ADAPTER_PORT}`);
console.log("[demo-gateway] No OpenClaw or Hermes required.");
});
}
if (require.main === module) {
startAdapter();
}
module.exports = {
handleMethod,
startAdapter,
};