feat: add SOUNDCLAW jukebox skill integration (#67)
Add the office jukebox flow so Spotify can be controlled from the SOUNDCLAW skill, manual jukebox UI, and local browser auth bridge during development. Made-with: Cursor
This commit is contained in:
@@ -82,3 +82,6 @@ test-results
|
||||
|
||||
# bv (beads viewer) local config and caches
|
||||
.bv/
|
||||
|
||||
# Local HTTPS development certificates (generated by dev:https).
|
||||
.certs/
|
||||
|
||||
@@ -193,6 +193,7 @@ See [`.env.example`](.env.example) for the full local development template.
|
||||
|
||||
- The immersive retro office (`/office`) and the Phaser builder (`/office/builder`) are related but still separate stacks.
|
||||
- The app keeps gateway secrets out of browser persistent storage, but the current connection flow still loads the upstream URL/token into browser memory at runtime.
|
||||
- Local Spotify auth for `SOUNDCLAW` currently stores an access token only. Refresh-token handling is not implemented yet, so local Spotify auth may need to be repeated after the token expires.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -205,6 +206,26 @@ If the UI loads but Connect fails, the problem is usually on the Studio -> Gatew
|
||||
|
||||
Marketplace skill installs now use a gateway-native workspace flow and do not require enabling SSH on the user machine.
|
||||
|
||||
### Spotify auth on localhost
|
||||
|
||||
If you are testing the `SOUNDCLAW` jukebox locally and Spotify OAuth does not accept your `localhost` callback, use an `ngrok` callback bridge:
|
||||
|
||||
1. Keep Claw3D running locally on `http://localhost:3000`.
|
||||
2. Start `ngrok` for the local Studio server, for example `ngrok http 3000`.
|
||||
3. In the jukebox setup UI, paste your public `ngrok` URL into the `ngrok Public URL` field.
|
||||
4. In the Spotify developer dashboard, register `https://<your-ngrok-host>/spotify/callback` as the redirect URI.
|
||||
5. Complete Spotify sign-in from the jukebox panel.
|
||||
|
||||
How it works:
|
||||
|
||||
- The main Claw3D app stays on `localhost`, so your normal local office state and agent state remain intact.
|
||||
- Spotify redirects to the `ngrok` callback URL.
|
||||
- The callback page passes the auth code back to the open local Claw3D window.
|
||||
|
||||
Current local limitation:
|
||||
|
||||
- Because only the Spotify access token is stored right now, you may need to repeat the `ngrok` auth flow when that token expires during local development.
|
||||
|
||||
If you use other advanced gateway-host operations over SSH:
|
||||
|
||||
- macOS: enable `System Settings` -> `General` -> `Sharing` -> `Remote Login`, and make sure the target user is allowed.
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
name: soundclaw
|
||||
description: Control Spotify playback, search music, and return shareable music links.
|
||||
metadata: {"openclaw":{"skillKey":"soundclaw"}}
|
||||
---
|
||||
|
||||
# SOUNDCLAW
|
||||
|
||||
Use this skill when the user wants an agent to search for music, play a song or playlist, control Spotify playback, or send back a shareable Spotify link on the same channel the request came from.
|
||||
|
||||
## Trigger
|
||||
|
||||
```json
|
||||
{
|
||||
"activation": {
|
||||
"anyPhrases": [
|
||||
"spotify",
|
||||
"play a song",
|
||||
"play this song",
|
||||
"play music",
|
||||
"play a playlist",
|
||||
"find a song",
|
||||
"queue this song",
|
||||
"music link"
|
||||
]
|
||||
},
|
||||
"movement": {
|
||||
"target": "jukebox",
|
||||
"skipIfAlreadyThere": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When this skill is activated, the agent should walk to the office jukebox before handling the request.
|
||||
|
||||
- Treat requests from Telegram or any other external surface as valid triggers when they ask for Spotify playback, search, queueing, or music-link sharing.
|
||||
- The physical behavior for this skill is: go to the jukebox, perform the music-selection workflow, then report the result.
|
||||
- If the agent is already at the jukebox, continue without adding extra movement narration.
|
||||
|
||||
## Channel behavior
|
||||
|
||||
- Reply on the same active channel or session that received the request.
|
||||
- If playback cannot start but a matching track, album, or playlist is found, send back the best Spotify link instead of failing silently.
|
||||
- If multiple matches are plausible, ask a clarifying question instead of guessing.
|
||||
|
||||
---
|
||||
|
||||
## OpenClaw Gateway Skill Contract
|
||||
|
||||
> This section is for developers implementing the backend skill handler in OpenClaw.
|
||||
> The Claw3D UI handles authentication via Spotify PKCE OAuth in the browser.
|
||||
> The gateway skill handles agent-driven requests via the `soundclaw.*` RPC namespace.
|
||||
|
||||
### Authentication model
|
||||
|
||||
The user authenticates directly in the browser (PKCE, no secret required).
|
||||
The access token is stored in browser `localStorage` under the key `soundclaw_token`.
|
||||
|
||||
For **agent-driven** playback (e.g. "play Jazz for me"), the gateway skill should either:
|
||||
- Use a server-side Spotify app token (Client Credentials) for search-only actions, or
|
||||
- Instruct the agent to tell the user to use the jukebox panel for actual playback
|
||||
|
||||
### RPC methods the gateway skill should expose
|
||||
|
||||
```ts
|
||||
// Search for tracks. Returns a list of { name, artist, album, uri, spotifyUrl }.
|
||||
soundclaw.search({ query: string }): SpotifySearchResult[]
|
||||
|
||||
// Get a shareable Spotify link for a query (for Telegram/chat replies).
|
||||
soundclaw.getLink({ query: string }): { url: string; title: string }
|
||||
|
||||
// Report current playback state (reads from Spotify API).
|
||||
soundclaw.playerStatus(): PlayerStatus | null
|
||||
|
||||
// Request playback of a URI (requires user to be authenticated in browser).
|
||||
soundclaw.play({ uri: string }): { ok: boolean; message?: string }
|
||||
|
||||
// Pause / resume / skip.
|
||||
soundclaw.pause(): void
|
||||
soundclaw.resume(): void
|
||||
soundclaw.next(): void
|
||||
soundclaw.previous(): void
|
||||
```
|
||||
|
||||
### Agent workflow
|
||||
|
||||
1. Agent receives a music request ("play some jazz", "find this song", etc.)
|
||||
2. Agent walks to the jukebox (`movement.target: "jukebox"`)
|
||||
3. Agent calls `soundclaw.search` to find the best match
|
||||
4. If the request came from a chat channel (Telegram, etc.): call `soundclaw.getLink` and reply with the link
|
||||
5. If the request came from the office UI: call `soundclaw.play` to start playback
|
||||
6. Agent reports back what was played or linked
|
||||
Generated
+277
@@ -41,6 +41,7 @@
|
||||
"eslint-config-next": "16.1.6",
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
"jsdom": "^27.4.0",
|
||||
"selfsigned": "^5.5.0",
|
||||
"tailwindcss": "^4",
|
||||
"tw-animate-css": "^1.4.0",
|
||||
"typescript": "^5",
|
||||
@@ -2090,6 +2091,165 @@
|
||||
"node": ">=14"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-cms": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-cms/-/asn1-cms-2.6.1.tgz",
|
||||
"integrity": "sha512-vdG4fBF6Lkirkcl53q6eOdn3XYKt+kJTG59edgRZORlg/3atWWEReRCx5rYE1ZzTTX6vLK5zDMjHh7vbrcXGtw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.1",
|
||||
"@peculiar/asn1-x509-attr": "^2.6.1",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-csr": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-csr/-/asn1-csr-2.6.1.tgz",
|
||||
"integrity": "sha512-WRWnKfIocHyzFYQTka8O/tXCiBquAPSrRjXbOkHbO4qdmS6loffCEGs+rby6WxxGdJCuunnhS2duHURhjyio6w==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.1",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-ecc": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-ecc/-/asn1-ecc-2.6.1.tgz",
|
||||
"integrity": "sha512-+Vqw8WFxrtDIN5ehUdvlN2m73exS2JVG0UAyfVB31gIfor3zWEAQPD+K9ydCxaj3MLen9k0JhKpu9LqviuCE1g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.1",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-pfx": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-pfx/-/asn1-pfx-2.6.1.tgz",
|
||||
"integrity": "sha512-nB5jVQy3MAAWvq0KY0R2JUZG8bO/bTLpnwyOzXyEh/e54ynGTatAR+csOnXkkVD9AFZ2uL8Z7EV918+qB1qDvw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-cms": "^2.6.1",
|
||||
"@peculiar/asn1-pkcs8": "^2.6.1",
|
||||
"@peculiar/asn1-rsa": "^2.6.1",
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-pkcs8": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-pkcs8/-/asn1-pkcs8-2.6.1.tgz",
|
||||
"integrity": "sha512-JB5iQ9Izn5yGMw3ZG4Nw3Xn/hb/G38GYF3lf7WmJb8JZUydhVGEjK/ZlFSWhnlB7K/4oqEs8HnfFIKklhR58Tw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.1",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-pkcs9": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-pkcs9/-/asn1-pkcs9-2.6.1.tgz",
|
||||
"integrity": "sha512-5EV8nZoMSxeWmcxWmmcolg22ojZRgJg+Y9MX2fnE2bGRo5KQLqV5IL9kdSQDZxlHz95tHvIq9F//bvL1OeNILw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-cms": "^2.6.1",
|
||||
"@peculiar/asn1-pfx": "^2.6.1",
|
||||
"@peculiar/asn1-pkcs8": "^2.6.1",
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.1",
|
||||
"@peculiar/asn1-x509-attr": "^2.6.1",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-rsa": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-rsa/-/asn1-rsa-2.6.1.tgz",
|
||||
"integrity": "sha512-1nVMEh46SElUt5CB3RUTV4EG/z7iYc7EoaDY5ECwganibQPkZ/Y2eMsTKB/LeyrUJ+W/tKoD9WUqIy8vB+CEdA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.1",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-schema": {
|
||||
"version": "2.6.0",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-schema/-/asn1-schema-2.6.0.tgz",
|
||||
"integrity": "sha512-xNLYLBFTBKkCzEZIw842BxytQQATQv+lDTCEMZ8C196iJcJJMBUZxrhSTxLaohMyKK8QlzRNTRkUmanucnDSqg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"asn1js": "^3.0.6",
|
||||
"pvtsutils": "^1.3.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-x509": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-x509/-/asn1-x509-2.6.1.tgz",
|
||||
"integrity": "sha512-O9jT5F1A2+t3r7C4VT7LYGXqkGLK7Kj1xFpz7U0isPrubwU5PbDoyYtx6MiGst29yq7pXN5vZbQFKRCP+lLZlA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"asn1js": "^3.0.6",
|
||||
"pvtsutils": "^1.3.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/asn1-x509-attr": {
|
||||
"version": "2.6.1",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/asn1-x509-attr/-/asn1-x509-attr-2.6.1.tgz",
|
||||
"integrity": "sha512-tlW6cxoHwgcQghnJwv3YS+9OO1737zgPogZ+CgWRUK4roEwIPzRH4JEiG770xe5HX2ATfCpmX60gurfWIF9dcQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.1",
|
||||
"asn1js": "^3.0.6",
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@peculiar/x509": {
|
||||
"version": "1.14.3",
|
||||
"resolved": "https://registry.npmjs.org/@peculiar/x509/-/x509-1.14.3.tgz",
|
||||
"integrity": "sha512-C2Xj8FZ0uHWeCXXqX5B4/gVFQmtSkiuOolzAgutjTfseNOHT3pUjljDZsTSxXFGgio54bCzVFqmEOUrIVk8RDA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/asn1-cms": "^2.6.0",
|
||||
"@peculiar/asn1-csr": "^2.6.0",
|
||||
"@peculiar/asn1-ecc": "^2.6.0",
|
||||
"@peculiar/asn1-pkcs9": "^2.6.0",
|
||||
"@peculiar/asn1-rsa": "^2.6.0",
|
||||
"@peculiar/asn1-schema": "^2.6.0",
|
||||
"@peculiar/asn1-x509": "^2.6.0",
|
||||
"pvtsutils": "^1.3.6",
|
||||
"reflect-metadata": "^0.2.2",
|
||||
"tslib": "^2.8.1",
|
||||
"tsyringe": "^4.10.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@playwright/test": {
|
||||
"version": "1.58.0",
|
||||
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.58.0.tgz",
|
||||
@@ -4083,6 +4243,21 @@
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/asn1js": {
|
||||
"version": "3.0.7",
|
||||
"resolved": "https://registry.npmjs.org/asn1js/-/asn1js-3.0.7.tgz",
|
||||
"integrity": "sha512-uLvq6KJu04qoQM6gvBfKFjlh6Gl0vOKQuR5cJMDHQkmwfMOQeN3F3SHCv9SNYSL+CRoHvOGFfllDlVz03GQjvQ==",
|
||||
"dev": true,
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"pvtsutils": "^1.3.6",
|
||||
"pvutils": "^1.1.3",
|
||||
"tslib": "^2.8.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/assertion-error": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz",
|
||||
@@ -4283,6 +4458,16 @@
|
||||
"ieee754": "^1.2.1"
|
||||
}
|
||||
},
|
||||
"node_modules/bytestreamjs": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/bytestreamjs/-/bytestreamjs-2.0.1.tgz",
|
||||
"integrity": "sha512-U1Z/ob71V/bXfVABvNr/Kumf5VyeQRBEm6Txb0PQ6S7V5GpBM3w4Cbqz/xPDicR5tN0uvDifng8C+5qECeGwyQ==",
|
||||
"dev": true,
|
||||
"license": "BSD-3-Clause",
|
||||
"engines": {
|
||||
"node": ">=6.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/call-bind": {
|
||||
"version": "1.0.8",
|
||||
"resolved": "https://registry.npmjs.org/call-bind/-/call-bind-1.0.8.tgz",
|
||||
@@ -8783,6 +8968,37 @@
|
||||
"url": "https://github.com/sponsors/jonschlinkert"
|
||||
}
|
||||
},
|
||||
"node_modules/pkijs": {
|
||||
"version": "3.4.0",
|
||||
"resolved": "https://registry.npmjs.org/pkijs/-/pkijs-3.4.0.tgz",
|
||||
"integrity": "sha512-emEcLuomt2j03vxD54giVB4SxTjnsqkU692xZOZXHDVoYyypEm+b3jpiTcc+Cf+myooc+/Ly0z01jqeNHVgJGw==",
|
||||
"dev": true,
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"@noble/hashes": "1.4.0",
|
||||
"asn1js": "^3.0.6",
|
||||
"bytestreamjs": "^2.0.1",
|
||||
"pvtsutils": "^1.3.6",
|
||||
"pvutils": "^1.1.3",
|
||||
"tslib": "^2.8.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=16.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/pkijs/node_modules/@noble/hashes": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-1.4.0.tgz",
|
||||
"integrity": "sha512-V1JJ1WTRUqHHrOSh597hURcMqVKVGL/ea3kv0gSnEdsEZ0/+VyPghM1lMNGc00z7CIQorSvbKpuJkxvuHbvdbg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 16"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://paulmillr.com/funding/"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright": {
|
||||
"version": "1.58.0",
|
||||
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.58.0.tgz",
|
||||
@@ -8950,6 +9166,26 @@
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/pvtsutils": {
|
||||
"version": "1.3.6",
|
||||
"resolved": "https://registry.npmjs.org/pvtsutils/-/pvtsutils-1.3.6.tgz",
|
||||
"integrity": "sha512-PLgQXQ6H2FWCaeRak8vvk1GW462lMxB5s3Jm673N82zI4vqtVUPuZdffdZbPDFRoU8kAhItWFtPCWiPpp4/EDg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"tslib": "^2.8.1"
|
||||
}
|
||||
},
|
||||
"node_modules/pvutils": {
|
||||
"version": "1.1.5",
|
||||
"resolved": "https://registry.npmjs.org/pvutils/-/pvutils-1.1.5.tgz",
|
||||
"integrity": "sha512-KTqnxsgGiQ6ZAzZCVlJH5eOjSnvlyEgx1m8bkRJfOhmGRqfo5KLvmAlACQkrjEtOQ4B7wF9TdSLIs9O90MX9xA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=16.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/queue-microtask": {
|
||||
"version": "1.2.3",
|
||||
"resolved": "https://registry.npmjs.org/queue-microtask/-/queue-microtask-1.2.3.tgz",
|
||||
@@ -9071,6 +9307,13 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/reflect-metadata": {
|
||||
"version": "0.2.2",
|
||||
"resolved": "https://registry.npmjs.org/reflect-metadata/-/reflect-metadata-0.2.2.tgz",
|
||||
"integrity": "sha512-urBwgfrvVP/eAyXx4hluJivBKzuEbSQs9rKWCrCkbSxNv8mxPcUZKeuoF3Uy4mJl3Lwprp6yy5/39VWigZ4K6Q==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/reflect.getprototypeof": {
|
||||
"version": "1.0.10",
|
||||
"resolved": "https://registry.npmjs.org/reflect.getprototypeof/-/reflect.getprototypeof-1.0.10.tgz",
|
||||
@@ -9399,6 +9642,20 @@
|
||||
"integrity": "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/selfsigned": {
|
||||
"version": "5.5.0",
|
||||
"resolved": "https://registry.npmjs.org/selfsigned/-/selfsigned-5.5.0.tgz",
|
||||
"integrity": "sha512-ftnu3TW4+3eBfLRFnDEkzGxSF/10BJBkaLJuBHZX0kiPS7bRdlpZGu6YGt4KngMkdTwJE6MbjavFpqHvqVt+Ew==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@peculiar/x509": "^1.14.2",
|
||||
"pkijs": "^3.3.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/semver": {
|
||||
"version": "6.3.1",
|
||||
"resolved": "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz",
|
||||
@@ -10244,6 +10501,26 @@
|
||||
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
|
||||
"license": "0BSD"
|
||||
},
|
||||
"node_modules/tsyringe": {
|
||||
"version": "4.10.0",
|
||||
"resolved": "https://registry.npmjs.org/tsyringe/-/tsyringe-4.10.0.tgz",
|
||||
"integrity": "sha512-axr3IdNuVIxnaK5XGEUFTu3YmAQ6lllgrvqfEoR16g/HGnYY/6We4oWENtAnzK6/LpJ2ur9PAb80RBt7/U4ugw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"tslib": "^1.9.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 6.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tsyringe/node_modules/tslib": {
|
||||
"version": "1.14.1",
|
||||
"resolved": "https://registry.npmjs.org/tslib/-/tslib-1.14.1.tgz",
|
||||
"integrity": "sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg==",
|
||||
"dev": true,
|
||||
"license": "0BSD"
|
||||
},
|
||||
"node_modules/tunnel-rat": {
|
||||
"version": "0.1.2",
|
||||
"resolved": "https://registry.npmjs.org/tunnel-rat/-/tunnel-rat-0.1.2.tgz",
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
"license": "MIT",
|
||||
"scripts": {
|
||||
"dev": "node server/index.js --dev",
|
||||
"dev:https": "node server/index.js --dev --https",
|
||||
"build": "next build",
|
||||
"start": "node server/index.js",
|
||||
"lint": "eslint .",
|
||||
@@ -49,6 +50,7 @@
|
||||
"eslint-config-next": "16.1.6",
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
"jsdom": "^27.4.0",
|
||||
"selfsigned": "^5.5.0",
|
||||
"tailwindcss": "^4",
|
||||
"tw-animate-css": "^1.4.0",
|
||||
"typescript": "^5",
|
||||
|
||||
+59
-2
@@ -1,4 +1,5 @@
|
||||
const http = require("node:http");
|
||||
const https = require("node:https");
|
||||
const next = require("next");
|
||||
|
||||
const { createAccessGate } = require("./access-gate");
|
||||
@@ -19,8 +20,52 @@ const resolvePathname = (url) => {
|
||||
return (idx === -1 ? raw : raw.slice(0, idx)) || "/";
|
||||
};
|
||||
|
||||
const CERT_DIR = require("node:path").join(__dirname, "..", ".certs");
|
||||
const CERT_PATH = require("node:path").join(CERT_DIR, "localhost.crt");
|
||||
const KEY_PATH = require("node:path").join(CERT_DIR, "localhost.key");
|
||||
|
||||
const generateHttpsCert = async () => {
|
||||
const fs = require("node:fs");
|
||||
|
||||
// Re-use a saved cert so the browser only needs to trust it once.
|
||||
if (fs.existsSync(CERT_PATH) && fs.existsSync(KEY_PATH)) {
|
||||
return {
|
||||
key: fs.readFileSync(KEY_PATH, "utf8"),
|
||||
cert: fs.readFileSync(CERT_PATH, "utf8"),
|
||||
};
|
||||
}
|
||||
|
||||
const selfsigned = require("selfsigned");
|
||||
const attrs = [{ name: "commonName", value: "localhost" }];
|
||||
const pems = await selfsigned.generate(attrs, {
|
||||
days: 825,
|
||||
keySize: 2048,
|
||||
algorithm: "sha256",
|
||||
extensions: [
|
||||
{
|
||||
name: "subjectAltName",
|
||||
altNames: [
|
||||
{ type: 2, value: "localhost" },
|
||||
{ type: 7, ip: "127.0.0.1" },
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
fs.mkdirSync(CERT_DIR, { recursive: true });
|
||||
fs.writeFileSync(CERT_PATH, pems.cert);
|
||||
fs.writeFileSync(KEY_PATH, pems.private);
|
||||
|
||||
console.info(`\nCert saved to ${CERT_DIR}`);
|
||||
console.info("To make browsers trust it (macOS), run:");
|
||||
console.info(` sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain "${CERT_PATH}"\n`);
|
||||
|
||||
return { key: pems.private, cert: pems.cert };
|
||||
};
|
||||
|
||||
async function main() {
|
||||
const dev = process.argv.includes("--dev");
|
||||
const useHttps = process.argv.includes("--https") || process.env.HTTPS === "true";
|
||||
const hostnames = Array.from(new Set(resolveHosts(process.env)));
|
||||
const hostname = hostnames[0] ?? "127.0.0.1";
|
||||
const port = resolvePort();
|
||||
@@ -65,8 +110,15 @@ async function main() {
|
||||
handleUpgrade(req, socket, head);
|
||||
};
|
||||
|
||||
const httpsCert = useHttps ? await generateHttpsCert() : null;
|
||||
|
||||
const createServer = () =>
|
||||
http.createServer((req, res) => {
|
||||
useHttps
|
||||
? https.createServer(httpsCert, (req, res) => {
|
||||
if (accessGate.handleHttp(req, res)) return;
|
||||
handle(req, res);
|
||||
})
|
||||
: http.createServer((req, res) => {
|
||||
if (accessGate.handleHttp(req, res)) return;
|
||||
handle(req, res);
|
||||
});
|
||||
@@ -120,8 +172,13 @@ async function main() {
|
||||
? "localhost"
|
||||
: hostname;
|
||||
|
||||
const browserUrl = `http://${hostForBrowser}:${port}`;
|
||||
const protocol = useHttps ? "https" : "http";
|
||||
const browserUrl = `${protocol}://${hostForBrowser}:${port}`;
|
||||
console.info(`Open in browser: ${browserUrl}`);
|
||||
if (useHttps) {
|
||||
console.info("HTTPS mode: self-signed cert in use. You may need to accept a browser security warning once.");
|
||||
console.info(`Spotify redirect URI: ${browserUrl}/office`);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect } from "react";
|
||||
|
||||
export default function SpotifyCallbackPage() {
|
||||
useEffect(() => {
|
||||
const url = new URL(window.location.href);
|
||||
const code = url.searchParams.get("code") ?? "";
|
||||
const state = url.searchParams.get("state") ?? "";
|
||||
const error = url.searchParams.get("error") ?? "";
|
||||
|
||||
if (window.opener && !window.opener.closed) {
|
||||
window.opener.postMessage(
|
||||
{
|
||||
type: "soundclaw-spotify-auth",
|
||||
code,
|
||||
state,
|
||||
error,
|
||||
},
|
||||
"*",
|
||||
);
|
||||
window.close();
|
||||
}
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<main className="flex min-h-screen items-center justify-center bg-slate-950 p-6 text-slate-100">
|
||||
<div className="w-full max-w-md rounded-3xl border border-cyan-500/20 bg-slate-900/90 p-8 text-center shadow-2xl">
|
||||
<div className="mb-2 font-mono text-[10px] uppercase tracking-[0.24em] text-cyan-300/70">
|
||||
Soundclaw
|
||||
</div>
|
||||
<h1 className="text-xl font-semibold text-white">Finishing Spotify sign-in</h1>
|
||||
<p className="mt-3 text-sm text-slate-400">
|
||||
You can close this window if it does not close automatically.
|
||||
</p>
|
||||
</div>
|
||||
</main>
|
||||
);
|
||||
}
|
||||
@@ -112,6 +112,13 @@ import { HistoryPanel } from "@/features/office/components/panels/HistoryPanel";
|
||||
import { InboxPanel } from "@/features/office/components/panels/InboxPanel";
|
||||
import { PlaybooksPanel } from "@/features/office/components/panels/PlaybooksPanel";
|
||||
import { SkillsMarketplaceModal } from "@/features/office/components/panels/SkillsMarketplaceModal";
|
||||
import { JukeboxPanel } from "@/features/spotify-jukebox/components/JukeboxPanel";
|
||||
import { JukeboxDisabledPanel } from "@/features/spotify-jukebox/components/JukeboxDisabledPanel";
|
||||
import { executeBrowserJukeboxCommand } from "@/features/spotify-jukebox/agentBridge";
|
||||
import {
|
||||
SOUNDCLAW_PLAYBACK_STARTED_EVENT_NAME,
|
||||
useJukeboxStore,
|
||||
} from "@/features/spotify-jukebox/store";
|
||||
import { useOfficeSkillTriggers } from "@/features/office/hooks/useOfficeSkillTriggers";
|
||||
import { useRemoteOfficePresence } from "@/features/office/hooks/useRemoteOfficePresence";
|
||||
import { useRemoteOfficeLayout } from "@/features/office/hooks/useRemoteOfficeLayout";
|
||||
@@ -147,6 +154,7 @@ import {
|
||||
buildOfficeDeskMonitor,
|
||||
type OfficeDeskMonitor,
|
||||
} from "@/lib/office/deskMonitor";
|
||||
import { deriveSkillReadinessState } from "@/lib/skills/presentation";
|
||||
import type { StandupAgentSnapshot } from "@/lib/office/standup/types";
|
||||
import type { SkillStatusEntry } from "@/lib/skills/types";
|
||||
|
||||
@@ -175,6 +183,31 @@ const GYM_WORKOUT_LATCH_MS = 60_000;
|
||||
const MAIN_AGENT_ID = "main";
|
||||
const MAX_OPENCLAW_LOG_ENTRIES = 200;
|
||||
const MAX_OPENCLAW_AGENT_OUTPUT_LINES = 12;
|
||||
const OFFICE_DANCE_MS = 60_000;
|
||||
|
||||
const getLatestUserRequestForAgent = (
|
||||
agent: AgentState,
|
||||
): { text: string; requestKey: string } | null => {
|
||||
const transcriptEntries = Array.isArray(agent.transcriptEntries)
|
||||
? agent.transcriptEntries
|
||||
: [];
|
||||
for (let index = transcriptEntries.length - 1; index >= 0; index -= 1) {
|
||||
const entry = transcriptEntries[index];
|
||||
if (!entry || entry.role !== "user") continue;
|
||||
const text = entry.text.trim();
|
||||
if (!text) continue;
|
||||
return {
|
||||
text,
|
||||
requestKey: `${agent.sessionKey}:${entry.sequenceKey}:${text}`,
|
||||
};
|
||||
}
|
||||
const fallback = agent.lastUserMessage?.trim() ?? "";
|
||||
if (!fallback) return null;
|
||||
return {
|
||||
text: fallback,
|
||||
requestKey: `${agent.sessionKey}:fallback:${fallback}`,
|
||||
};
|
||||
};
|
||||
|
||||
type OpenClawLogEntry = {
|
||||
id: string;
|
||||
@@ -890,12 +923,67 @@ export function OfficeScreen({
|
||||
const [gatewayModels, setGatewayModels] = useState<GatewayModelChoice[]>([]);
|
||||
const [sidebarOpen, setSidebarOpen] = useState(false);
|
||||
const [marketplaceOpen, setMarketplaceOpen] = useState(false);
|
||||
const [danceUntilByAgentId, setDanceUntilByAgentId] = useState<Record<string, number>>({});
|
||||
const initJukeboxStore = useJukeboxStore((state) => state.init);
|
||||
const jukeboxToken = useJukeboxStore((state) => state.token);
|
||||
// Auto-open jukebox panel for legacy direct-auth callbacks.
|
||||
const [jukeboxOpen, setJukeboxOpen] = useState(() => {
|
||||
if (typeof window === "undefined") return false;
|
||||
const searchParams = new URL(window.location.href).searchParams;
|
||||
return searchParams.has("code");
|
||||
});
|
||||
const [activeSidebarTab, setActiveSidebarTab] =
|
||||
useState<HQSidebarTab>("inbox");
|
||||
const pendingJukeboxCommandTimeoutsRef = useRef<
|
||||
Map<string, { requestKey: string; timeoutId: number }>
|
||||
>(new Map());
|
||||
const handledJukeboxRequestKeyByAgentIdRef = useRef<Record<string, string>>({});
|
||||
const router = useRouter();
|
||||
const { showOnboarding, completeOnboarding, resetOnboarding } =
|
||||
useOnboardingState();
|
||||
const [forceShowOnboarding, setForceShowOnboarding] = useState(false);
|
||||
useEffect(() => {
|
||||
initJukeboxStore();
|
||||
}, [initJukeboxStore]);
|
||||
useEffect(() => {
|
||||
const handlePlaybackStarted = () => {
|
||||
const now = Date.now();
|
||||
const until = now + OFFICE_DANCE_MS;
|
||||
setDanceUntilByAgentId((previous) => {
|
||||
const next: Record<string, number> = {};
|
||||
for (const agent of state.agents) {
|
||||
next[agent.agentId] = until;
|
||||
}
|
||||
return { ...previous, ...next };
|
||||
});
|
||||
};
|
||||
window.addEventListener(
|
||||
SOUNDCLAW_PLAYBACK_STARTED_EVENT_NAME,
|
||||
handlePlaybackStarted,
|
||||
);
|
||||
return () => {
|
||||
window.removeEventListener(
|
||||
SOUNDCLAW_PLAYBACK_STARTED_EVENT_NAME,
|
||||
handlePlaybackStarted,
|
||||
);
|
||||
};
|
||||
}, [state.agents]);
|
||||
useEffect(() => {
|
||||
const now = Date.now();
|
||||
setDanceUntilByAgentId((previous) =>
|
||||
Object.fromEntries(
|
||||
Object.entries(previous).filter(([, until]) => until > now),
|
||||
),
|
||||
);
|
||||
}, [state.agents]);
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
for (const pendingEntry of pendingJukeboxCommandTimeoutsRef.current.values()) {
|
||||
window.clearTimeout(pendingEntry.timeoutId);
|
||||
}
|
||||
pendingJukeboxCommandTimeoutsRef.current.clear();
|
||||
};
|
||||
}, []);
|
||||
const {
|
||||
loaded: officeTitleLoaded,
|
||||
title: officeTitle,
|
||||
@@ -2245,6 +2333,7 @@ export function OfficeScreen({
|
||||
|
||||
return {
|
||||
...base,
|
||||
danceUntilByAgentId: danceUntilByAgentId,
|
||||
deskHoldByAgentId: {
|
||||
...base.deskHoldByAgentId,
|
||||
...skillTriggerHoldMaps.deskHoldByAgentId,
|
||||
@@ -2257,6 +2346,10 @@ export function OfficeScreen({
|
||||
...base.gymHoldByAgentId,
|
||||
...skillTriggerHoldMaps.gymHoldByAgentId,
|
||||
},
|
||||
jukeboxHoldByAgentId: {
|
||||
...base.jukeboxHoldByAgentId,
|
||||
...skillTriggerHoldMaps.jukeboxHoldByAgentId,
|
||||
},
|
||||
qaHoldByAgentId: {
|
||||
...base.qaHoldByAgentId,
|
||||
...skillTriggerHoldMaps.qaHoldByAgentId,
|
||||
@@ -2268,6 +2361,7 @@ export function OfficeScreen({
|
||||
};
|
||||
}, [
|
||||
animationNowMs,
|
||||
danceUntilByAgentId,
|
||||
marketplaceGymHoldByAgentId,
|
||||
officeTriggerState,
|
||||
skillTriggers.movementTargetByAgentId,
|
||||
@@ -2276,6 +2370,7 @@ export function OfficeScreen({
|
||||
const {
|
||||
deskHoldByAgentId,
|
||||
githubHoldByAgentId,
|
||||
jukeboxHoldByAgentId,
|
||||
manualGymUntilByAgentId,
|
||||
pendingStandupRequest,
|
||||
phoneBoothHoldByAgentId,
|
||||
@@ -3453,6 +3548,109 @@ export function OfficeScreen({
|
||||
}) ?? null,
|
||||
[marketplace.skillsReport],
|
||||
);
|
||||
const soundclawSkill = useMemo<SkillStatusEntry | null>(
|
||||
() =>
|
||||
marketplace.skillsReport?.skills.find((skill) => {
|
||||
const normalizedKey = skill.skillKey.trim().toLowerCase();
|
||||
const normalizedName = skill.name.trim().toLowerCase();
|
||||
return normalizedKey === "soundclaw" || normalizedName === "soundclaw";
|
||||
}) ?? null,
|
||||
[marketplace.skillsReport],
|
||||
);
|
||||
const soundclawReady = useMemo(
|
||||
() => (soundclawSkill ? deriveSkillReadinessState(soundclawSkill) === "ready" : false),
|
||||
[soundclawSkill]
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!soundclawReady || !jukeboxToken) {
|
||||
return;
|
||||
}
|
||||
|
||||
const pending = pendingJukeboxCommandTimeoutsRef.current;
|
||||
const activeAgentIds = new Set<string>();
|
||||
|
||||
for (const agent of state.agents) {
|
||||
if (skillTriggers.movementTargetByAgentId[agent.agentId] !== "jukebox") {
|
||||
continue;
|
||||
}
|
||||
|
||||
const request = getLatestUserRequestForAgent(agent);
|
||||
if (!request) {
|
||||
continue;
|
||||
}
|
||||
|
||||
activeAgentIds.add(agent.agentId);
|
||||
const handledKey = handledJukeboxRequestKeyByAgentIdRef.current[agent.agentId];
|
||||
if (handledKey === request.requestKey) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const existing = pending.get(agent.agentId);
|
||||
if (existing?.requestKey === request.requestKey) {
|
||||
continue;
|
||||
}
|
||||
if (existing) {
|
||||
window.clearTimeout(existing.timeoutId);
|
||||
pending.delete(agent.agentId);
|
||||
}
|
||||
|
||||
const timeoutId = window.setTimeout(() => {
|
||||
void executeBrowserJukeboxCommand(request.text).then((result) => {
|
||||
if (result.ok) {
|
||||
handledJukeboxRequestKeyByAgentIdRef.current[agent.agentId] = request.requestKey;
|
||||
setJukeboxOpen(true);
|
||||
dispatch({
|
||||
type: "appendOutput",
|
||||
agentId: agent.agentId,
|
||||
line: result.reply,
|
||||
transcript: {
|
||||
role: "assistant",
|
||||
kind: "assistant",
|
||||
source: "legacy",
|
||||
sessionKey: agent.sessionKey,
|
||||
timestampMs: Date.now(),
|
||||
confirmed: true,
|
||||
},
|
||||
});
|
||||
dispatch({
|
||||
type: "updateAgent",
|
||||
agentId: agent.agentId,
|
||||
patch: {
|
||||
latestOverride: result.reply,
|
||||
latestOverrideKind: null,
|
||||
latestPreview: result.reply,
|
||||
lastAssistantMessageAt: Date.now(),
|
||||
},
|
||||
});
|
||||
}
|
||||
const latest = pendingJukeboxCommandTimeoutsRef.current.get(agent.agentId);
|
||||
if (latest?.timeoutId === timeoutId) {
|
||||
pendingJukeboxCommandTimeoutsRef.current.delete(agent.agentId);
|
||||
}
|
||||
});
|
||||
}, 1400);
|
||||
|
||||
pending.set(agent.agentId, {
|
||||
requestKey: request.requestKey,
|
||||
timeoutId,
|
||||
});
|
||||
}
|
||||
|
||||
for (const [agentId, pendingEntry] of pending.entries()) {
|
||||
if (activeAgentIds.has(agentId)) continue;
|
||||
window.clearTimeout(pendingEntry.timeoutId);
|
||||
pending.delete(agentId);
|
||||
}
|
||||
}, [
|
||||
jukeboxToken,
|
||||
skillTriggers.movementTargetByAgentId,
|
||||
soundclawReady,
|
||||
state.agents,
|
||||
]);
|
||||
|
||||
// No longer force-close the jukebox panel when skill is disabled;
|
||||
// the panel handles the disabled state itself.
|
||||
|
||||
if (
|
||||
!agentsLoaded &&
|
||||
@@ -3497,6 +3695,7 @@ export function OfficeScreen({
|
||||
agent.status === "running" ||
|
||||
deskHoldByAgentId[agent.agentId] ||
|
||||
gymHoldByAgentId[agent.agentId] ||
|
||||
jukeboxHoldByAgentId[agent.agentId] ||
|
||||
phoneBoothHoldByAgentId[agent.agentId] ||
|
||||
smsBoothHoldByAgentId[agent.agentId] ||
|
||||
qaHoldByAgentId[agent.agentId],
|
||||
@@ -3526,6 +3725,7 @@ export function OfficeScreen({
|
||||
monitorAgentId={monitorAgentId}
|
||||
monitorByAgentId={monitorByAgentId}
|
||||
githubSkill={githubSkill}
|
||||
soundclawEnabled={soundclawReady}
|
||||
officeTitle={officeTitle}
|
||||
officeTitleLoaded={officeTitleLoaded}
|
||||
remoteOfficeEnabled={remoteOfficeEnabled}
|
||||
@@ -3615,7 +3815,27 @@ export function OfficeScreen({
|
||||
onOpenGithubSkillSetup={() => {
|
||||
setMarketplaceOpen(true);
|
||||
}}
|
||||
onJukeboxInteract={() => {
|
||||
setJukeboxOpen(true);
|
||||
}}
|
||||
/>
|
||||
{jukeboxOpen ? (
|
||||
soundclawReady ? (
|
||||
<JukeboxPanel
|
||||
client={client}
|
||||
onClose={() => setJukeboxOpen(false)}
|
||||
selectedAgentName={focusedChatAgent?.name ?? null}
|
||||
/>
|
||||
) : (
|
||||
<JukeboxDisabledPanel
|
||||
onClose={() => setJukeboxOpen(false)}
|
||||
onInstall={() => {
|
||||
setJukeboxOpen(false);
|
||||
setMarketplaceOpen(true);
|
||||
}}
|
||||
/>
|
||||
)
|
||||
) : null}
|
||||
</section>
|
||||
|
||||
{showEmptyFleetBanner ? (
|
||||
|
||||
@@ -76,6 +76,7 @@ import {
|
||||
ensureOfficePingPongTable,
|
||||
ensureOfficeQaLab,
|
||||
ensureOfficeSmsBooth,
|
||||
ensureOfficeJukebox,
|
||||
ensureOfficeServerRoom,
|
||||
isRetiredPingPongLamp,
|
||||
materializeDefaults,
|
||||
@@ -148,6 +149,7 @@ import type {
|
||||
import type { NavGrid } from "@/features/retro-office/core/navigation";
|
||||
import type { OfficeLayoutSnapshot } from "@/lib/office/layoutSnapshot";
|
||||
import { AgentModel as AgentObjectModel } from "@/features/retro-office/objects/agents";
|
||||
import { JukeboxModel as InteractiveJukeboxModel } from "@/features/retro-office/objects/Jukebox";
|
||||
import {
|
||||
FurnitureModel as GenericFurnitureModel,
|
||||
InstancedFurnitureItems as InstancedFurnitureItemsModel,
|
||||
@@ -359,6 +361,7 @@ const PALETTE: PaletteEntry[] = [
|
||||
{ type: "fridge", label: "Fridge", icon: "🧊", defaults: { w: 40, h: 80 } },
|
||||
{ type: "water_cooler", label: "Water", icon: "💧", defaults: {} },
|
||||
{ type: "atm", label: "ATM", icon: "🏧", defaults: { facing: 270 } },
|
||||
{ type: "jukebox", label: "Jukebox", icon: "🎵", defaults: { facing: 0 } },
|
||||
{
|
||||
type: "whiteboard",
|
||||
label: "Whiteboard",
|
||||
@@ -403,11 +406,7 @@ const PALETTE: PaletteEntry[] = [
|
||||
// CAMERA SETUP — sets lookAt after mount
|
||||
// ============================================================
|
||||
|
||||
function CameraRig({
|
||||
target,
|
||||
}: {
|
||||
target: [number, number, number];
|
||||
}) {
|
||||
function CameraRig({ target }: { target: [number, number, number] }) {
|
||||
const { camera } = useThree();
|
||||
useEffect(() => {
|
||||
camera.lookAt(...target);
|
||||
@@ -444,8 +443,9 @@ const ReadOnlyFurnitureClone = memo(function ReadOnlyFurnitureClone({
|
||||
<InstancedFurnitureItemsModel itemType="desk_cubicle" items={deskItems} />
|
||||
<InstancedFurnitureItemsModel itemType="chair" items={chairItems} />
|
||||
{furniture.map((item) =>
|
||||
item.type === "wall" || item.type === "desk_cubicle" || item.type === "chair" ? null
|
||||
: item.type === "door" ? (
|
||||
item.type === "wall" ||
|
||||
item.type === "desk_cubicle" ||
|
||||
item.type === "chair" ? null : item.type === "door" ? (
|
||||
<PrimitiveDoorModel
|
||||
key={item._uid}
|
||||
item={item}
|
||||
@@ -846,6 +846,7 @@ function useAgentTick(
|
||||
furnitureRef: React.RefObject<FurnitureItem[]>,
|
||||
lastSeenByAgentId: Record<string, number> = {},
|
||||
deskHoldByAgentId: Record<string, boolean> = {},
|
||||
danceUntilByAgentId: Record<string, number> = {},
|
||||
gymHoldByAgentId: Record<string, boolean> = {},
|
||||
smsBoothHoldByAgentId: Record<string, boolean> = {},
|
||||
phoneBoothHoldByAgentId: Record<string, boolean> = {},
|
||||
@@ -882,13 +883,17 @@ function useAgentTick(
|
||||
);
|
||||
const pickRoamPoint = useCallback((agentId: string) => {
|
||||
if (isRemoteOfficeAgentId(agentId)) {
|
||||
return REMOTE_ROAM_POINTS[Math.floor(Math.random() * REMOTE_ROAM_POINTS.length)];
|
||||
return REMOTE_ROAM_POINTS[
|
||||
Math.floor(Math.random() * REMOTE_ROAM_POINTS.length)
|
||||
];
|
||||
}
|
||||
return ROAM_POINTS[Math.floor(Math.random() * ROAM_POINTS.length)];
|
||||
}, []);
|
||||
const pickSpawnPoint = useCallback((agentId: string) => {
|
||||
if (isRemoteOfficeAgentId(agentId)) {
|
||||
return REMOTE_ROAM_POINTS[Math.floor(Math.random() * REMOTE_ROAM_POINTS.length)];
|
||||
return REMOTE_ROAM_POINTS[
|
||||
Math.floor(Math.random() * REMOTE_ROAM_POINTS.length)
|
||||
];
|
||||
}
|
||||
return {
|
||||
x: Math.random() * 800 + 100,
|
||||
@@ -1046,11 +1051,13 @@ function useAgentTick(
|
||||
? resolveMeetingTarget(agent.id)
|
||||
: null;
|
||||
const smsBoothItem =
|
||||
(furnitureRef.current ?? []).find((item) => item.type === "sms_booth") ??
|
||||
null;
|
||||
(furnitureRef.current ?? []).find(
|
||||
(item) => item.type === "sms_booth",
|
||||
) ?? null;
|
||||
const phoneBoothItem =
|
||||
(furnitureRef.current ?? []).find((item) => item.type === "phone_booth") ??
|
||||
null;
|
||||
(furnitureRef.current ?? []).find(
|
||||
(item) => item.type === "phone_booth",
|
||||
) ?? null;
|
||||
|
||||
if (agent.status === "working" && !explicitDeskHold && deskPos)
|
||||
stickyUntilRef.current.set(agent.id, now + DESK_STICKY_MS);
|
||||
@@ -1501,9 +1508,7 @@ function useAgentTick(
|
||||
? undefined
|
||||
: phoneBoothRoute.stage;
|
||||
ns.smsBoothStage =
|
||||
explicitMeetingHold ||
|
||||
explicitGymHold ||
|
||||
!explicitSmsBoothHold
|
||||
explicitMeetingHold || explicitGymHold || !explicitSmsBoothHold
|
||||
? undefined
|
||||
: smsBoothRoute.stage;
|
||||
ns.serverRoomStage = explicitMeetingHold
|
||||
@@ -2132,6 +2137,11 @@ function useAgentTick(
|
||||
}
|
||||
}
|
||||
|
||||
if ((danceUntilByAgentId[agent.id] ?? 0) > now && ns !== "away") {
|
||||
ns = "dancing";
|
||||
npath = [];
|
||||
}
|
||||
|
||||
return {
|
||||
...agent,
|
||||
x: nx,
|
||||
@@ -2163,7 +2173,8 @@ function useAgentTick(
|
||||
if ("role" in mi && mi.role === "janitor") continue;
|
||||
if (
|
||||
moved[i].state === "sitting" ||
|
||||
moved[i].state === "working_out"
|
||||
moved[i].state === "working_out" ||
|
||||
moved[i].state === "dancing"
|
||||
)
|
||||
continue;
|
||||
if (moved[i].pingPongUntil !== undefined && moved[i].state !== "walking")
|
||||
@@ -2178,7 +2189,9 @@ function useAgentTick(
|
||||
const bucketY = Math.floor(mi.y / collisionCellSize);
|
||||
for (let offsetY = -1; offsetY <= 1; offsetY += 1) {
|
||||
for (let offsetX = -1; offsetX <= 1; offsetX += 1) {
|
||||
const bucket = collisionBuckets.get(`${bucketX + offsetX}:${bucketY + offsetY}`);
|
||||
const bucket = collisionBuckets.get(
|
||||
`${bucketX + offsetX}:${bucketY + offsetY}`,
|
||||
);
|
||||
if (!bucket) continue;
|
||||
for (const j of bucket) {
|
||||
if (i === j) continue;
|
||||
@@ -2246,7 +2259,13 @@ function useAgentTick(
|
||||
}
|
||||
};
|
||||
|
||||
return { renderAgentsRef, renderAgentLookupRef, tick, deskByAgentRef, planPath };
|
||||
return {
|
||||
renderAgentsRef,
|
||||
renderAgentLookupRef,
|
||||
tick,
|
||||
deskByAgentRef,
|
||||
planPath,
|
||||
};
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
@@ -2256,7 +2275,9 @@ function useAgentTick(
|
||||
const AWAY_THRESHOLD_MS = 15 * 60 * 1000;
|
||||
const COMPACT_AGENT_BADGE_LIMIT = 6;
|
||||
|
||||
const estimatePhoneSpeechDurationMs = (text: string | null | undefined): number => {
|
||||
const estimatePhoneSpeechDurationMs = (
|
||||
text: string | null | undefined,
|
||||
): number => {
|
||||
const normalized = text?.trim() ?? "";
|
||||
if (!normalized) return 5_000;
|
||||
const wordCount = normalized.split(/\s+/).filter(Boolean).length;
|
||||
@@ -2293,6 +2314,7 @@ export function RetroOffice3D({
|
||||
monitorAgentId = null,
|
||||
monitorByAgentId = EMPTY_MONITOR_MAP,
|
||||
githubSkill = null,
|
||||
soundclawEnabled = false,
|
||||
officeTitle = "Luke Headquarters",
|
||||
officeTitleLoaded = false,
|
||||
remoteOfficeEnabled = false,
|
||||
@@ -2340,17 +2362,20 @@ export function RetroOffice3D({
|
||||
onTextMessageComplete,
|
||||
onQaLabDismiss,
|
||||
onOpenGithubSkillSetup,
|
||||
onJukeboxInteract,
|
||||
}: {
|
||||
agents: OfficeAgent[];
|
||||
animationState?: Pick<
|
||||
OfficeAnimationState,
|
||||
| "cleaningCues"
|
||||
| "danceUntilByAgentId"
|
||||
| "deskHoldByAgentId"
|
||||
| "githubHoldByAgentId"
|
||||
| "gymHoldByAgentId"
|
||||
| "phoneBoothHoldByAgentId"
|
||||
| "smsBoothHoldByAgentId"
|
||||
| "qaHoldByAgentId"
|
||||
| "jukeboxHoldByAgentId"
|
||||
> | null;
|
||||
readOnly?: boolean;
|
||||
storageNamespace?: string;
|
||||
@@ -2370,6 +2395,7 @@ export function RetroOffice3D({
|
||||
monitorAgentId?: string | null;
|
||||
monitorByAgentId?: OfficeDeskMonitorMap;
|
||||
githubSkill?: SkillStatusEntry | null;
|
||||
soundclawEnabled?: boolean;
|
||||
officeTitle?: string;
|
||||
officeTitleLoaded?: boolean;
|
||||
remoteOfficeEnabled?: boolean;
|
||||
@@ -2386,7 +2412,9 @@ export function RetroOffice3D({
|
||||
voiceRepliesLoaded?: boolean;
|
||||
onOfficeTitleChange?: (title: string) => void;
|
||||
onRemoteOfficeEnabledChange?: (enabled: boolean) => void;
|
||||
onRemoteOfficeSourceKindChange?: (kind: "presence_endpoint" | "openclaw_gateway") => void;
|
||||
onRemoteOfficeSourceKindChange?: (
|
||||
kind: "presence_endpoint" | "openclaw_gateway",
|
||||
) => void;
|
||||
onRemoteOfficeLabelChange?: (label: string) => void;
|
||||
onRemoteOfficePresenceUrlChange?: (url: string) => void;
|
||||
onRemoteOfficeGatewayUrlChange?: (url: string) => void;
|
||||
@@ -2421,8 +2449,11 @@ export function RetroOffice3D({
|
||||
onTextMessageComplete?: (agentId: string) => void;
|
||||
onQaLabDismiss?: () => void;
|
||||
onOpenGithubSkillSetup?: () => void;
|
||||
onJukeboxInteract?: () => void;
|
||||
}) {
|
||||
const resolvedCleaningCues = animationState?.cleaningCues ?? cleaningCues;
|
||||
const resolvedDanceUntilByAgentId =
|
||||
animationState?.danceUntilByAgentId ?? EMPTY_NUMBER_RECORD;
|
||||
const resolvedDeskHoldByAgentId =
|
||||
animationState?.deskHoldByAgentId ?? deskHoldByAgentId;
|
||||
const resolvedGymHoldByAgentId =
|
||||
@@ -2438,7 +2469,12 @@ export function RetroOffice3D({
|
||||
(githubReviewAgentId
|
||||
? { [githubReviewAgentId]: true }
|
||||
: EMPTY_BOOLEAN_RECORD);
|
||||
const resolvedJukeboxHoldByAgentId =
|
||||
animationState?.jukeboxHoldByAgentId ?? EMPTY_BOOLEAN_RECORD;
|
||||
const isJukeboxActive = Object.values(resolvedJukeboxHoldByAgentId).some(Boolean);
|
||||
|
||||
const [furniture, setFurniture] = useState<FurnitureItem[]>(() =>
|
||||
ensureOfficeJukebox(
|
||||
ensureOfficeQaLab(
|
||||
ensureOfficeGymRoom(
|
||||
ensureOfficeServerRoom(
|
||||
@@ -2446,8 +2482,9 @@ export function RetroOffice3D({
|
||||
ensureOfficeSmsBooth(
|
||||
ensureOfficeAtm(
|
||||
ensureOfficePingPongTable(
|
||||
(loadFurniture(storageNamespace) ?? materializeDefaults()).filter(
|
||||
(item) => !isRetiredPingPongLamp(item),
|
||||
(
|
||||
loadFurniture(storageNamespace) ?? materializeDefaults()
|
||||
).filter((item) => !isRetiredPingPongLamp(item)),
|
||||
),
|
||||
),
|
||||
),
|
||||
@@ -2572,8 +2609,10 @@ export function RetroOffice3D({
|
||||
const [monitorImmersiveReady, setMonitorImmersiveReady] = useState(false);
|
||||
const [activeAtmUid, setActiveAtmUid] = useState<string | null>(null);
|
||||
const [atmImmersiveReady, setAtmImmersiveReady] = useState(false);
|
||||
const [phoneBoothCommandArrived, setPhoneBoothCommandArrived] = useState(false);
|
||||
const [phoneBoothImmersiveReady, setPhoneBoothImmersiveReady] = useState(false);
|
||||
const [phoneBoothCommandArrived, setPhoneBoothCommandArrived] =
|
||||
useState(false);
|
||||
const [phoneBoothImmersiveReady, setPhoneBoothImmersiveReady] =
|
||||
useState(false);
|
||||
const [phoneBoothDoorOpen, setPhoneBoothDoorOpen] = useState(false);
|
||||
const [phoneCallStep, setPhoneCallStep] = useState<PhoneCallStep>("dialing");
|
||||
const [dialedDigits, setDialedDigits] = useState("");
|
||||
@@ -2585,9 +2624,9 @@ export function RetroOffice3D({
|
||||
const [typedMessageText, setTypedMessageText] = useState("");
|
||||
const [activeTextKey, setActiveTextKey] = useState<string | null>(null);
|
||||
const [textContacts, setTextContacts] = useState<string[]>([]);
|
||||
const [activeTextContactIndex, setActiveTextContactIndex] = useState<number | null>(
|
||||
null,
|
||||
);
|
||||
const [activeTextContactIndex, setActiveTextContactIndex] = useState<
|
||||
number | null
|
||||
>(null);
|
||||
const [manualPhoneBoothOpen, setManualPhoneBoothOpen] = useState(false);
|
||||
const [manualPhoneCallScenario, setManualPhoneCallScenario] =
|
||||
useState<MockPhoneCallScenario | null>(null);
|
||||
@@ -2598,14 +2637,17 @@ export function RetroOffice3D({
|
||||
const activeTextMessageFlowKeyRef = useRef<string | null>(null);
|
||||
const boothAudioCtxRef = useRef<AudioContext | null>(null);
|
||||
const effectivePhoneBoothAgentIdRef = useRef<string | null>(null);
|
||||
const effectivePhoneCallScenarioRef = useRef<MockPhoneCallScenario | null>(null);
|
||||
const effectivePhoneCallScenarioRef = useRef<MockPhoneCallScenario | null>(
|
||||
null,
|
||||
);
|
||||
const phoneBoothAgentIdRef = useRef<string | null>(null);
|
||||
const onPhoneCallSpeakRef = useRef(onPhoneCallSpeak);
|
||||
const onPhoneCallCompleteRef = useRef(onPhoneCallComplete);
|
||||
const onStandupArrivalsChangeRef = useRef(onStandupArrivalsChange);
|
||||
const lastStandupArrivalKeyRef = useRef<string | null>(null);
|
||||
const effectiveSmsBoothAgentIdRef = useRef<string | null>(null);
|
||||
const effectiveTextMessageScenarioRef = useRef<MockTextMessageScenario | null>(null);
|
||||
const effectiveTextMessageScenarioRef =
|
||||
useRef<MockTextMessageScenario | null>(null);
|
||||
const smsBoothAgentIdRef = useRef<string | null>(null);
|
||||
const onTextMessageCompleteRef = useRef(onTextMessageComplete);
|
||||
const [activeGithubTerminalUid, setActiveGithubTerminalUid] = useState<
|
||||
@@ -2732,8 +2774,13 @@ export function RetroOffice3D({
|
||||
[agents, janitorActors],
|
||||
);
|
||||
|
||||
const { renderAgentsRef, renderAgentLookupRef, tick, deskByAgentRef, planPath } =
|
||||
useAgentTick(
|
||||
const {
|
||||
renderAgentsRef,
|
||||
renderAgentLookupRef,
|
||||
tick,
|
||||
deskByAgentRef,
|
||||
planPath,
|
||||
} = useAgentTick(
|
||||
sceneAgents,
|
||||
deskLocations,
|
||||
assignedDeskIndexByAgentId,
|
||||
@@ -2743,6 +2790,7 @@ export function RetroOffice3D({
|
||||
furnitureRef,
|
||||
lastSeenByAgentId,
|
||||
resolvedDeskHoldByAgentId,
|
||||
resolvedDanceUntilByAgentId,
|
||||
resolvedGymHoldByAgentId,
|
||||
resolvedSmsBoothHoldByAgentId,
|
||||
resolvedPhoneBoothHoldByAgentId,
|
||||
@@ -2773,31 +2821,42 @@ export function RetroOffice3D({
|
||||
: null;
|
||||
const agentStatusLookup = useMemo(
|
||||
() =>
|
||||
agents.reduce<Record<string, { isError: boolean; working: boolean }>>((acc, agent) => {
|
||||
agents.reduce<Record<string, { isError: boolean; working: boolean }>>(
|
||||
(acc, agent) => {
|
||||
const renderAgent = renderAgentUiById[agent.id];
|
||||
acc[agent.id] = {
|
||||
isError: renderAgent?.status === "error" || agent.status === "error",
|
||||
isError:
|
||||
renderAgent?.status === "error" || agent.status === "error",
|
||||
working:
|
||||
renderAgent?.state === "sitting" ||
|
||||
renderAgent?.state === "dancing" ||
|
||||
renderAgent?.status === "working" ||
|
||||
agent.status === "working",
|
||||
};
|
||||
return acc;
|
||||
}, {}),
|
||||
},
|
||||
{},
|
||||
),
|
||||
[agents, renderAgentUiById],
|
||||
);
|
||||
const hoveredAgent = useMemo(
|
||||
() => (hoveredAgentId ? agents.find((agent) => agent.id === hoveredAgentId) ?? null : null),
|
||||
() =>
|
||||
hoveredAgentId
|
||||
? (agents.find((agent) => agent.id === hoveredAgentId) ?? null)
|
||||
: null,
|
||||
[agents, hoveredAgentId],
|
||||
);
|
||||
const hoveredAgentStatus = hoveredAgentId ? agentStatusLookup[hoveredAgentId] ?? null : null;
|
||||
const hoveredAgentStatus = hoveredAgentId
|
||||
? (agentStatusLookup[hoveredAgentId] ?? null)
|
||||
: null;
|
||||
const handleAgentHover = useCallback((agentId: string) => {
|
||||
setHoveredAgentId(agentId);
|
||||
}, []);
|
||||
const handleAgentUnhover = useCallback(() => {
|
||||
setHoveredAgentId(null);
|
||||
}, []);
|
||||
const handleAgentClick = useCallback((agentId: string) => {
|
||||
const handleAgentClick = useCallback(
|
||||
(agentId: string) => {
|
||||
const agent = renderAgentLookupRef.current.get(agentId);
|
||||
if (!agent || !orbitRef.current) return;
|
||||
const [wx, , wz] = toWorld(agent.x, agent.y);
|
||||
@@ -2806,11 +2865,16 @@ export function RetroOffice3D({
|
||||
if (isRemoteOfficeAgentId(agentId)) {
|
||||
onAgentChatSelect?.(agentId);
|
||||
}
|
||||
}, [onAgentChatSelect, renderAgentLookupRef]);
|
||||
const handleAgentContextMenu = useCallback((agentId: string, x: number, y: number) => {
|
||||
},
|
||||
[onAgentChatSelect, renderAgentLookupRef],
|
||||
);
|
||||
const handleAgentContextMenu = useCallback(
|
||||
(agentId: string, x: number, y: number) => {
|
||||
if (isRemoteOfficeAgentId(agentId)) return;
|
||||
setContextMenu({ id: agentId, x, y });
|
||||
}, []);
|
||||
},
|
||||
[],
|
||||
);
|
||||
const monitorImmersive = Boolean(activeMonitor && monitorImmersiveReady);
|
||||
const serverTerminal = useMemo(
|
||||
() => furniture.find((item) => item.type === "server_terminal") ?? null,
|
||||
@@ -2847,11 +2911,14 @@ export function RetroOffice3D({
|
||||
[furniture],
|
||||
);
|
||||
const effectivePhoneCallScenario =
|
||||
phoneCallScenario ?? (manualPhoneBoothOpen ? manualPhoneCallScenario : null);
|
||||
phoneCallScenario ??
|
||||
(manualPhoneBoothOpen ? manualPhoneCallScenario : null);
|
||||
const effectivePhoneBoothAgentId =
|
||||
phoneBoothAgentId ?? (manualPhoneBoothOpen ? "__manual_phone_booth__" : null);
|
||||
phoneBoothAgentId ??
|
||||
(manualPhoneBoothOpen ? "__manual_phone_booth__" : null);
|
||||
const phoneBoothViewActive =
|
||||
manualPhoneBoothOpen || Boolean(phoneBoothAgentId && phoneBoothCommandArrived);
|
||||
manualPhoneBoothOpen ||
|
||||
Boolean(phoneBoothAgentId && phoneBoothCommandArrived);
|
||||
const activePhoneCallFlowKey = useMemo(() => {
|
||||
if (!effectivePhoneBoothAgentId || !effectivePhoneCallScenario) return null;
|
||||
return [
|
||||
@@ -2869,7 +2936,8 @@ export function RetroOffice3D({
|
||||
phoneBoothImmersiveReady,
|
||||
);
|
||||
const effectiveTextMessageScenario =
|
||||
textMessageScenario ?? (manualSmsBoothOpen ? manualTextMessageScenario : null);
|
||||
textMessageScenario ??
|
||||
(manualSmsBoothOpen ? manualTextMessageScenario : null);
|
||||
const effectiveSmsBoothAgentId =
|
||||
smsBoothAgentId ?? (manualSmsBoothOpen ? "__manual_sms_booth__" : null);
|
||||
const smsBoothViewActive =
|
||||
@@ -2949,7 +3017,10 @@ export function RetroOffice3D({
|
||||
() => agents.slice(0, COMPACT_AGENT_BADGE_LIMIT),
|
||||
[agents],
|
||||
);
|
||||
const hiddenAgentCount = Math.max(0, agents.length - compactRosterAgents.length);
|
||||
const hiddenAgentCount = Math.max(
|
||||
0,
|
||||
agents.length - compactRosterAgents.length,
|
||||
);
|
||||
const standupActive =
|
||||
standupMeeting?.phase === "gathering" ||
|
||||
standupMeeting?.phase === "in_progress";
|
||||
@@ -3196,7 +3267,11 @@ export function RetroOffice3D({
|
||||
}, [getBoothAudioContext]);
|
||||
|
||||
const playTextKeyTone = useCallback(
|
||||
async (options?: { frequency?: number; durationMs?: number; gain?: number }) => {
|
||||
async (options?: {
|
||||
frequency?: number;
|
||||
durationMs?: number;
|
||||
gain?: number;
|
||||
}) => {
|
||||
const audioContext = await getBoothAudioContext();
|
||||
if (!audioContext) return;
|
||||
const now = audioContext.currentTime;
|
||||
@@ -3271,7 +3346,9 @@ export function RetroOffice3D({
|
||||
await audioContext.resume();
|
||||
}
|
||||
const arrayBuffer = await blob.arrayBuffer();
|
||||
const decoded = await audioContext.decodeAudioData(arrayBuffer.slice(0));
|
||||
const decoded = await audioContext.decodeAudioData(
|
||||
arrayBuffer.slice(0),
|
||||
);
|
||||
const source = audioContext.createBufferSource();
|
||||
source.buffer = decoded;
|
||||
source.connect(audioContext.destination);
|
||||
@@ -3457,7 +3534,9 @@ export function RetroOffice3D({
|
||||
agent.qaLabStage === "station" &&
|
||||
Math.hypot(agent.x - agent.targetX, agent.y - agent.targetY) < 16,
|
||||
);
|
||||
setQaCommandArrived((current) => (current === arrived ? current : arrived));
|
||||
setQaCommandArrived((current) =>
|
||||
current === arrived ? current : arrived,
|
||||
);
|
||||
}
|
||||
|
||||
if (!phoneBoothAgentId) {
|
||||
@@ -3509,7 +3588,9 @@ export function RetroOffice3D({
|
||||
setSmsBoothCommandArrived((current) =>
|
||||
current === arrived ? current : arrived,
|
||||
);
|
||||
setSmsBoothDoorOpen((current) => (current === doorOpen ? current : doorOpen));
|
||||
setSmsBoothDoorOpen((current) =>
|
||||
current === doorOpen ? current : doorOpen,
|
||||
);
|
||||
}
|
||||
|
||||
if (!standupActive || !standupMeeting) {
|
||||
@@ -3520,11 +3601,16 @@ export function RetroOffice3D({
|
||||
return;
|
||||
}
|
||||
|
||||
const arrivedParticipants = standupMeeting.participantOrder.filter((agentId) => {
|
||||
const arrivedParticipants = standupMeeting.participantOrder.filter(
|
||||
(agentId) => {
|
||||
const agent = agentLookup.get(agentId);
|
||||
if (!agent || agent.interactionTarget !== "meeting_room") return false;
|
||||
return Math.hypot(agent.x - agent.targetX, agent.y - agent.targetY) < 18;
|
||||
});
|
||||
if (!agent || agent.interactionTarget !== "meeting_room")
|
||||
return false;
|
||||
return (
|
||||
Math.hypot(agent.x - agent.targetX, agent.y - agent.targetY) < 18
|
||||
);
|
||||
},
|
||||
);
|
||||
const nextArrivalsKey = arrivedParticipants.join("|");
|
||||
if (lastStandupArrivalKeyRef.current === nextArrivalsKey) return;
|
||||
lastStandupArrivalKeyRef.current = nextArrivalsKey;
|
||||
@@ -3667,7 +3753,10 @@ export function RetroOffice3D({
|
||||
if (pressedKey) {
|
||||
pulseKeyboardKey(pressedKey);
|
||||
}
|
||||
if (index >= (scenario.messageText?.length ?? 0) && typingTimer !== null) {
|
||||
if (
|
||||
index >= (scenario.messageText?.length ?? 0) &&
|
||||
typingTimer !== null
|
||||
) {
|
||||
window.clearInterval(typingTimer);
|
||||
typingTimer = null;
|
||||
clearActiveKey();
|
||||
@@ -3758,7 +3847,12 @@ export function RetroOffice3D({
|
||||
zoom: 228,
|
||||
};
|
||||
prevSmsBoothViewRef.current = activeViewKey;
|
||||
}, [activeSmsBooth, manualSmsBoothOpen, smsBoothAgentId, smsBoothCommandArrived]);
|
||||
}, [
|
||||
activeSmsBooth,
|
||||
manualSmsBoothOpen,
|
||||
smsBoothAgentId,
|
||||
smsBoothCommandArrived,
|
||||
]);
|
||||
|
||||
useEffect(() => {
|
||||
const resetTimer = window.setTimeout(() => {
|
||||
@@ -3842,7 +3936,9 @@ export function RetroOffice3D({
|
||||
setPhoneCallStep("complete");
|
||||
stageTimer = window.setTimeout(() => {
|
||||
if (phoneBoothAgentIdRef.current) {
|
||||
onPhoneCallCompleteRef.current?.(phoneBoothAgentIdRef.current);
|
||||
onPhoneCallCompleteRef.current?.(
|
||||
phoneBoothAgentIdRef.current,
|
||||
);
|
||||
} else {
|
||||
closeManualPhoneBoothView();
|
||||
}
|
||||
@@ -3876,8 +3972,7 @@ export function RetroOffice3D({
|
||||
]);
|
||||
|
||||
useEffect(() => {
|
||||
const activeViewKey =
|
||||
manualPhoneBoothOpen
|
||||
const activeViewKey = manualPhoneBoothOpen
|
||||
? "manual"
|
||||
: phoneBoothAgentId && phoneBoothCommandArrived
|
||||
? `agent:${phoneBoothAgentId}`
|
||||
@@ -3900,7 +3995,12 @@ export function RetroOffice3D({
|
||||
zoom: 210,
|
||||
};
|
||||
prevPhoneBoothViewRef.current = activeViewKey;
|
||||
}, [activePhoneBooth, manualPhoneBoothOpen, phoneBoothAgentId, phoneBoothCommandArrived]);
|
||||
}, [
|
||||
activePhoneBooth,
|
||||
manualPhoneBoothOpen,
|
||||
phoneBoothAgentId,
|
||||
phoneBoothCommandArrived,
|
||||
]);
|
||||
|
||||
useEffect(() => {
|
||||
const resetTimer = window.setTimeout(() => {
|
||||
@@ -4850,10 +4950,13 @@ export function RetroOffice3D({
|
||||
// Camera constants.
|
||||
const CAM_POS = DISTRICT_CAMERA_POSITION;
|
||||
const LOCAL_CAMERA_TARGET = useMemo(
|
||||
() => toWorld(LOCAL_OFFICE_CANVAS_WIDTH / 2, LOCAL_OFFICE_CANVAS_HEIGHT / 2),
|
||||
() =>
|
||||
toWorld(LOCAL_OFFICE_CANVAS_WIDTH / 2, LOCAL_OFFICE_CANVAS_HEIGHT / 2),
|
||||
[],
|
||||
);
|
||||
const cameraTarget = remoteOfficeEnabled ? DISTRICT_CAMERA_TARGET : LOCAL_CAMERA_TARGET;
|
||||
const cameraTarget = remoteOfficeEnabled
|
||||
? DISTRICT_CAMERA_TARGET
|
||||
: LOCAL_CAMERA_TARGET;
|
||||
const cameraZoom = remoteOfficeEnabled ? DISTRICT_CAMERA_ZOOM : 56;
|
||||
|
||||
return (
|
||||
@@ -4883,7 +4986,12 @@ export function RetroOffice3D({
|
||||
<Canvas
|
||||
orthographic
|
||||
dpr={[0.85, 1.5]}
|
||||
camera={{ position: CAM_POS, zoom: cameraZoom, near: 0.1, far: 100 }}
|
||||
camera={{
|
||||
position: CAM_POS,
|
||||
zoom: cameraZoom,
|
||||
near: 0.1,
|
||||
far: 100,
|
||||
}}
|
||||
shadows={{ type: THREE.PCFShadowMap }}
|
||||
gl={{ antialias: true, powerPreference: "high-performance" }}
|
||||
style={{ width: "100%", height: "100%" }}
|
||||
@@ -4979,7 +5087,10 @@ export function RetroOffice3D({
|
||||
/>
|
||||
) : null}
|
||||
{!editMode ? (
|
||||
<InstancedFurnitureItemsModel itemType="chair" items={chairItems} />
|
||||
<InstancedFurnitureItemsModel
|
||||
itemType="chair"
|
||||
items={chairItems}
|
||||
/>
|
||||
) : null}
|
||||
{furniture.map((item) =>
|
||||
item.type === "wall" ? (
|
||||
@@ -5104,6 +5215,20 @@ export function RetroOffice3D({
|
||||
onPointerOut={handleFurniturePointerOut}
|
||||
onClick={handleDeskClick}
|
||||
/>
|
||||
) : item.type === "jukebox" ? (
|
||||
<InteractiveJukeboxModel
|
||||
key={item._uid}
|
||||
item={item}
|
||||
active={isJukeboxActive}
|
||||
enabled={soundclawEnabled}
|
||||
isSelected={item._uid === selectedUid}
|
||||
isHovered={item._uid === hoverUid}
|
||||
editMode={editMode}
|
||||
onPointerDown={handleFurniturePointerDown}
|
||||
onPointerOver={handleFurniturePointerOver}
|
||||
onPointerOut={handleFurniturePointerOut}
|
||||
onClick={editMode ? handleDeskClick : () => onJukeboxInteract?.()}
|
||||
/>
|
||||
) : item.type === "sms_booth" ? (
|
||||
<InteractiveSmsBoothModel
|
||||
key={item._uid}
|
||||
@@ -5372,6 +5497,8 @@ export function RetroOffice3D({
|
||||
<ReadOnlyFurnitureClone furniture={remoteLayoutFurniture} />
|
||||
) : null}
|
||||
|
||||
{/* Removed standalone Jukebox as it's now in the furniture loop */}
|
||||
|
||||
{/* Agents — purely imperative, driven by renderAgentsRef inside useFrame. */}
|
||||
{sceneAgents.map((agent) => {
|
||||
const isJanitor = "role" in agent && agent.role === "janitor";
|
||||
@@ -5382,7 +5509,11 @@ export function RetroOffice3D({
|
||||
name={agent.name}
|
||||
status={agent.status}
|
||||
color={agentColorMap.get(agent.id) ?? "#888"}
|
||||
appearance={"avatarProfile" in agent ? agent.avatarProfile ?? null : null}
|
||||
appearance={
|
||||
"avatarProfile" in agent
|
||||
? (agent.avatarProfile ?? null)
|
||||
: null
|
||||
}
|
||||
agentsRef={renderAgentsRef}
|
||||
agentLookupRef={renderAgentLookupRef}
|
||||
onHover={isJanitor ? undefined : handleAgentHover}
|
||||
@@ -5511,7 +5642,11 @@ export function RetroOffice3D({
|
||||
icon: <Monitor size={12} />,
|
||||
title: "Front desk",
|
||||
},
|
||||
{ key: "lounge", icon: <Armchair size={12} />, title: "Lounge" },
|
||||
{
|
||||
key: "lounge",
|
||||
icon: <Armchair size={12} />,
|
||||
title: "Lounge",
|
||||
},
|
||||
] as const
|
||||
).map(({ key, icon, title }) => (
|
||||
<button
|
||||
@@ -5603,7 +5738,9 @@ export function RetroOffice3D({
|
||||
<span
|
||||
key={mood.ts}
|
||||
className="absolute -top-6 left-1/2 -translate-x-1/2 text-sm pointer-events-none"
|
||||
style={{ animation: "mood-float 2.5s ease-out forwards" }}
|
||||
style={{
|
||||
animation: "mood-float 2.5s ease-out forwards",
|
||||
}}
|
||||
>
|
||||
{mood.emoji}
|
||||
</span>
|
||||
@@ -5709,10 +5846,14 @@ export function RetroOffice3D({
|
||||
<button
|
||||
type="button"
|
||||
title={
|
||||
followAgentId === agent.id ? "Exit follow cam" : "Follow cam"
|
||||
followAgentId === agent.id
|
||||
? "Exit follow cam"
|
||||
: "Follow cam"
|
||||
}
|
||||
onClick={() =>
|
||||
setFollowAgentId((prev) => (prev === agent.id ? null : agent.id))
|
||||
setFollowAgentId((prev) =>
|
||||
prev === agent.id ? null : agent.id,
|
||||
)
|
||||
}
|
||||
className={`flex h-8 w-8 items-center justify-center rounded-lg border transition-colors ${
|
||||
followAgentId === agent.id
|
||||
@@ -5734,7 +5875,9 @@ export function RetroOffice3D({
|
||||
disabled={isRemoteAgent}
|
||||
onClick={() => {
|
||||
if (!isRemoteAgent) {
|
||||
onMonitorSelect?.(monitorAgentId === agent.id ? null : agent.id);
|
||||
onMonitorSelect?.(
|
||||
monitorAgentId === agent.id ? null : agent.id,
|
||||
);
|
||||
}
|
||||
}}
|
||||
className={`flex h-8 w-8 items-center justify-center rounded-lg border transition-colors ${
|
||||
@@ -5773,8 +5916,10 @@ export function RetroOffice3D({
|
||||
{!immersiveOverlayActive &&
|
||||
hoveredAgent &&
|
||||
(() => {
|
||||
const isError = hoveredAgentStatus?.isError ?? hoveredAgent.status === "error";
|
||||
const working = hoveredAgentStatus?.working ?? hoveredAgent.status === "working";
|
||||
const isError =
|
||||
hoveredAgentStatus?.isError ?? hoveredAgent.status === "error";
|
||||
const working =
|
||||
hoveredAgentStatus?.working ?? hoveredAgent.status === "working";
|
||||
return (
|
||||
<div className="absolute top-24 left-1/2 -translate-x-1/2 z-30 pointer-events-none select-none">
|
||||
<div className="flex items-center gap-3 bg-[#120e08]/95 backdrop-blur-sm border border-amber-800/30 rounded-lg px-4 py-2.5 shadow-xl">
|
||||
@@ -6219,7 +6364,9 @@ export function RetroOffice3D({
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
{smsBoothImmersive && effectiveTextMessageScenario && effectiveSmsBoothAgentId ? (
|
||||
{smsBoothImmersive &&
|
||||
effectiveTextMessageScenario &&
|
||||
effectiveSmsBoothAgentId ? (
|
||||
<div className="pointer-events-none absolute inset-0 z-20 overflow-hidden">
|
||||
<div className="absolute inset-0 bg-black/45" />
|
||||
<div className="absolute inset-x-0 top-0 h-[8vh] bg-[linear-gradient(180deg,rgba(0,0,0,0.94),rgba(0,0,0,0.62))]" />
|
||||
@@ -6259,7 +6406,9 @@ export function RetroOffice3D({
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
{phoneBoothImmersive && effectivePhoneCallScenario && effectivePhoneBoothAgentId ? (
|
||||
{phoneBoothImmersive &&
|
||||
effectivePhoneCallScenario &&
|
||||
effectivePhoneBoothAgentId ? (
|
||||
<div className="pointer-events-none absolute inset-0 z-20 overflow-hidden">
|
||||
<div className="absolute inset-0 bg-black/45" />
|
||||
<div className="absolute inset-x-0 top-0 h-[8vh] bg-[linear-gradient(180deg,rgba(0,0,0,0.94),rgba(0,0,0,0.62))]" />
|
||||
|
||||
@@ -53,6 +53,13 @@ const DEFAULT_SMS_BOOTH: FurnitureSeed = {
|
||||
facing: 0,
|
||||
};
|
||||
|
||||
const DEFAULT_JUKEBOX: FurnitureSeed = {
|
||||
type: "jukebox",
|
||||
x: 20,
|
||||
y: 380,
|
||||
facing: 90,
|
||||
};
|
||||
|
||||
const PREVIOUS_SERVER_ROOM_ITEMS_BOTTOM_RIGHT: FurnitureSeed[] = [
|
||||
{ type: "wall", x: 820, y: 540, w: 280, h: WALL_THICKNESS },
|
||||
{ type: "wall", x: 820, y: 540, w: WALL_THICKNESS, h: 70 },
|
||||
@@ -167,8 +174,7 @@ const LEGACY_QA_LAB_ITEMS: FurnitureSeed[] = [
|
||||
];
|
||||
|
||||
const EAST_WING_ROOM_BOTTOM_Y = EAST_WING_ROOM_TOP_Y + EAST_WING_ROOM_HEIGHT;
|
||||
const EAST_WING_ROOM_BOTTOM_WALL_Y =
|
||||
EAST_WING_ROOM_BOTTOM_Y - WALL_THICKNESS;
|
||||
const EAST_WING_ROOM_BOTTOM_WALL_Y = EAST_WING_ROOM_BOTTOM_Y - WALL_THICKNESS;
|
||||
const EAST_WING_DOOR_BOTTOM_Y = EAST_WING_DOOR_Y + DOOR_LENGTH;
|
||||
const EAST_WING_TOP_WALL_HEIGHT = EAST_WING_DOOR_Y - EAST_WING_ROOM_TOP_Y;
|
||||
const EAST_WING_BOTTOM_WALL_HEIGHT =
|
||||
@@ -558,15 +564,10 @@ const QA_LAB_SIGNATURES = new Set(
|
||||
DEFAULT_QA_LAB_ITEMS.map(createFurnitureSignature),
|
||||
);
|
||||
|
||||
const hasSignature = (
|
||||
items: FurnitureItem[],
|
||||
signatures: Set<string>,
|
||||
) => items.some((item) => signatures.has(createFurnitureSignature(item)));
|
||||
const hasSignature = (items: FurnitureItem[], signatures: Set<string>) =>
|
||||
items.some((item) => signatures.has(createFurnitureSignature(item)));
|
||||
|
||||
const hasAllSignatures = (
|
||||
items: FurnitureItem[],
|
||||
signatures: Set<string>,
|
||||
) => {
|
||||
const hasAllSignatures = (items: FurnitureItem[], signatures: Set<string>) => {
|
||||
const itemSignatures = new Set(items.map(createFurnitureSignature));
|
||||
return [...signatures].every((signature) => itemSignatures.has(signature));
|
||||
};
|
||||
@@ -574,8 +575,7 @@ const hasAllSignatures = (
|
||||
const replaceBySignatureSet = (
|
||||
items: FurnitureItem[],
|
||||
signatures: Set<string>,
|
||||
) =>
|
||||
items.filter((item) => !signatures.has(createFurnitureSignature(item)));
|
||||
) => items.filter((item) => !signatures.has(createFurnitureSignature(item)));
|
||||
|
||||
export const ensureOfficePingPongTable = (
|
||||
items: FurnitureItem[],
|
||||
@@ -590,7 +590,14 @@ export const ensureOfficeAtm = (items: FurnitureItem[]): FurnitureItem[] => {
|
||||
return [...items, { ...DEFAULT_ATM_MACHINE, _uid: nextUid() }];
|
||||
};
|
||||
|
||||
export const ensureOfficePhoneBooth = (items: FurnitureItem[]): FurnitureItem[] => {
|
||||
export const ensureOfficeJukebox = (items: FurnitureItem[]): FurnitureItem[] => {
|
||||
if (items.some((item) => item.type === "jukebox")) return items;
|
||||
return [...items, { ...DEFAULT_JUKEBOX, _uid: nextUid() }];
|
||||
};
|
||||
|
||||
export const ensureOfficePhoneBooth = (
|
||||
items: FurnitureItem[],
|
||||
): FurnitureItem[] => {
|
||||
let found = false;
|
||||
const nextItems = items.map((item) => {
|
||||
if (item.type === "phone_booth") {
|
||||
@@ -607,7 +614,9 @@ export const ensureOfficePhoneBooth = (items: FurnitureItem[]): FurnitureItem[]
|
||||
return [...nextItems, { ...DEFAULT_PHONE_BOOTH, _uid: nextUid() }];
|
||||
};
|
||||
|
||||
export const ensureOfficeSmsBooth = (items: FurnitureItem[]): FurnitureItem[] => {
|
||||
export const ensureOfficeSmsBooth = (
|
||||
items: FurnitureItem[],
|
||||
): FurnitureItem[] => {
|
||||
if (items.some((item) => item.type === "sms_booth")) return items;
|
||||
if (hasSmsBoothMigrationApplied()) return items;
|
||||
return [...items, { ...DEFAULT_SMS_BOOTH, _uid: nextUid() }];
|
||||
@@ -666,7 +675,10 @@ export const ensureOfficeGymRoom = (
|
||||
const hasCurrentGymRoom = hasSignature(items, GYM_ROOM_SIGNATURES);
|
||||
if (hasCurrentGymRoom) return items;
|
||||
|
||||
const hasPreviousGymRoom = hasAllSignatures(items, PREVIOUS_GYM_ROOM_SIGNATURES);
|
||||
const hasPreviousGymRoom = hasAllSignatures(
|
||||
items,
|
||||
PREVIOUS_GYM_ROOM_SIGNATURES,
|
||||
);
|
||||
if (hasPreviousGymRoom) {
|
||||
return [
|
||||
...replaceBySignatureSet(items, PREVIOUS_GYM_ROOM_SIGNATURES),
|
||||
@@ -733,4 +745,3 @@ export const ensureOfficeQaLab = (items: FurnitureItem[]): FurnitureItem[] => {
|
||||
...DEFAULT_QA_LAB_ITEMS.map((item) => ({ ...item, _uid: nextUid() })),
|
||||
];
|
||||
};
|
||||
|
||||
|
||||
@@ -74,6 +74,7 @@ export const ITEM_FOOTPRINT: Record<string, [number, number]> = {
|
||||
dumbbell_rack: [80, 28],
|
||||
exercise_bike: [45, 65],
|
||||
punching_bag: [28, 28],
|
||||
jukebox: [60, 40],
|
||||
rowing_machine: [90, 34],
|
||||
kettlebell_rack: [70, 26],
|
||||
yoga_mat: [70, 30],
|
||||
@@ -156,6 +157,7 @@ export const ITEM_METADATA: Record<string, { blocksNavigation: boolean }> = {
|
||||
dumbbell_rack: { blocksNavigation: true },
|
||||
exercise_bike: { blocksNavigation: true },
|
||||
punching_bag: { blocksNavigation: true },
|
||||
jukebox: { blocksNavigation: true },
|
||||
rowing_machine: { blocksNavigation: true },
|
||||
kettlebell_rack: { blocksNavigation: true },
|
||||
yoga_mat: { blocksNavigation: true },
|
||||
|
||||
@@ -1,7 +1,4 @@
|
||||
import {
|
||||
CANVAS_H,
|
||||
CANVAS_W,
|
||||
} from "@/features/retro-office/core/constants";
|
||||
import { CANVAS_H, CANVAS_W } from "@/features/retro-office/core/constants";
|
||||
import {
|
||||
getItemBounds,
|
||||
ITEM_FOOTPRINT,
|
||||
@@ -277,8 +274,10 @@ export function astar(
|
||||
// agents cannot clip through the corner of a blocked cell (issue #6).
|
||||
// E.g. moving NE (dc=+1, dr=-1) requires N (dc=0, dr=-1) and E (dc=+1, dr=0) to be clear.
|
||||
if (columnOffset !== 0 && rowOffset !== 0) {
|
||||
const orthogonalA = (currentRow + rowOffset) * GRID_COLS + currentColumn;
|
||||
const orthogonalB = currentRow * GRID_COLS + (currentColumn + columnOffset);
|
||||
const orthogonalA =
|
||||
(currentRow + rowOffset) * GRID_COLS + currentColumn;
|
||||
const orthogonalB =
|
||||
currentRow * GRID_COLS + (currentColumn + columnOffset);
|
||||
if (grid[orthogonalA] || grid[orthogonalB]) continue;
|
||||
}
|
||||
const nextCost = gCost[current] + cost;
|
||||
|
||||
@@ -37,7 +37,7 @@ export type RenderAgent = SceneActor & {
|
||||
frame: number;
|
||||
walkSpeed: number;
|
||||
phaseOffset: number;
|
||||
state: "walking" | "sitting" | "standing" | "away" | "working_out";
|
||||
state: "walking" | "sitting" | "standing" | "away" | "working_out" | "dancing";
|
||||
awayUntil?: number;
|
||||
separationReplanAt?: number;
|
||||
bumpedUntil?: number;
|
||||
|
||||
@@ -0,0 +1,228 @@
|
||||
"use client";
|
||||
|
||||
import { Billboard, Text } from "@react-three/drei";
|
||||
import { useFrame } from "@react-three/fiber";
|
||||
import { useRef, useState } from "react";
|
||||
import * as THREE from "three";
|
||||
import { SCALE } from "@/features/retro-office/core/constants";
|
||||
import {
|
||||
getItemBaseSize,
|
||||
getItemRotationRadians,
|
||||
toWorld,
|
||||
} from "@/features/retro-office/core/geometry";
|
||||
import type { InteractiveFurnitureModelProps } from "@/features/retro-office/objects/types";
|
||||
|
||||
export type JukeboxModelProps = InteractiveFurnitureModelProps & {
|
||||
active?: boolean;
|
||||
/** False when the soundclaw skill is not installed. */
|
||||
enabled?: boolean;
|
||||
};
|
||||
|
||||
const C = {
|
||||
cabinet: "#0d9488",
|
||||
cabinetDark: "#0f766e",
|
||||
metal: "#e2e8f0",
|
||||
metalDark: "#94a3b8",
|
||||
neon: "#FF1493",
|
||||
neonActive: "#00FF00",
|
||||
display: "#042f2e",
|
||||
displayText: "#00FF00",
|
||||
record: "#1a1a1a",
|
||||
recordLabel: "#FF1493",
|
||||
};
|
||||
|
||||
const BUTTON_COLORS = ["#FF0000", "#FFFF00", "#00FF00", "#00FFFF", "#FF00FF"];
|
||||
|
||||
export function JukeboxModel({
|
||||
item,
|
||||
isSelected,
|
||||
isHovered,
|
||||
active = false,
|
||||
enabled = true,
|
||||
onPointerDown,
|
||||
onPointerOver,
|
||||
onPointerOut,
|
||||
onClick,
|
||||
}: JukeboxModelProps) {
|
||||
const [localHovered, setLocalHovered] = useState(false);
|
||||
const recordRef = useRef<THREE.Mesh>(null);
|
||||
const glowRef = useRef<THREE.PointLight>(null);
|
||||
|
||||
const [wx, , wz] = toWorld(item.x, item.y);
|
||||
const { width, height } = getItemBaseSize(item);
|
||||
const rotY = getItemRotationRadians(item);
|
||||
|
||||
// Scale the model so it fills the furniture footprint.
|
||||
const scaleX = (width * SCALE) / 0.9;
|
||||
const scaleZ = (height * SCALE) / 0.7;
|
||||
|
||||
const highlighted = isSelected || isHovered;
|
||||
const playing = active && enabled;
|
||||
|
||||
// When the skill isn't installed, desaturate everything to grey.
|
||||
const tint = (enabledColor: string, disabledColor: string) =>
|
||||
enabled ? enabledColor : disabledColor;
|
||||
|
||||
useFrame((_state, delta) => {
|
||||
if (recordRef.current) {
|
||||
recordRef.current.rotation.y += playing ? delta * 2 : delta * 0.3;
|
||||
}
|
||||
if (glowRef.current && playing) {
|
||||
const pulse = Math.sin(_state.clock.elapsedTime * 4) * 0.3 + 0.7;
|
||||
glowRef.current.intensity = pulse * 2;
|
||||
}
|
||||
});
|
||||
|
||||
return (
|
||||
<group
|
||||
position={[wx, 0, wz]}
|
||||
onPointerDown={(e) => { e.stopPropagation(); onPointerDown(item._uid); }}
|
||||
onPointerOver={(e) => { e.stopPropagation(); setLocalHovered(true); onPointerOver(item._uid); document.body.style.cursor = "pointer"; }}
|
||||
onPointerOut={(e) => { e.stopPropagation(); setLocalHovered(false); onPointerOut(); document.body.style.cursor = ""; }}
|
||||
onClick={(e) => { e.stopPropagation(); onClick?.(item._uid); }}
|
||||
>
|
||||
<group rotation={[0, rotY, 0]} scale={[scaleX, 1, scaleZ]}>
|
||||
|
||||
{/* Main cabinet body. */}
|
||||
<mesh position={[0, 0.75, 0]} castShadow receiveShadow>
|
||||
<boxGeometry args={[0.8, 1.2, 0.6]} />
|
||||
<meshStandardMaterial
|
||||
color={tint(highlighted ? "#0f9a8e" : C.cabinet, highlighted ? "#555" : "#444")}
|
||||
roughness={0.6}
|
||||
metalness={0.1}
|
||||
/>
|
||||
</mesh>
|
||||
|
||||
{/* Cabinet top dome (tapered cylinder). */}
|
||||
<mesh position={[0, 1.4, 0]} castShadow>
|
||||
<cylinderGeometry args={[0.45, 0.5, 0.2, 32]} />
|
||||
<meshStandardMaterial color={tint(C.cabinetDark, "#333")} roughness={0.5} metalness={0.2} />
|
||||
</mesh>
|
||||
|
||||
{/* Chrome dome cap. */}
|
||||
<mesh position={[0, 1.55, 0]} castShadow>
|
||||
<sphereGeometry args={[0.15, 16, 16, 0, Math.PI * 2, 0, Math.PI / 2]} />
|
||||
<meshStandardMaterial color={tint(C.metal, "#666")} roughness={0.3} metalness={0.8} />
|
||||
</mesh>
|
||||
|
||||
{/* Display screen. */}
|
||||
<mesh position={[0, 1.1, 0.31]}>
|
||||
<planeGeometry args={[0.6, 0.35]} />
|
||||
<meshStandardMaterial
|
||||
color={tint(C.display, "#1a1a1a")}
|
||||
emissive={enabled ? (playing ? C.neonActive : C.neon) : "#333"}
|
||||
emissiveIntensity={enabled ? (localHovered || isHovered ? 0.5 : 0.2) : 0.08}
|
||||
/>
|
||||
</mesh>
|
||||
|
||||
{/* Track status / disabled text on display. */}
|
||||
<Billboard position={[0, 1.1, 0.32]} follow={false}>
|
||||
<Text
|
||||
fontSize={0.07}
|
||||
color={enabled ? C.displayText : "#666"}
|
||||
anchorX="center"
|
||||
anchorY="middle"
|
||||
maxWidth={0.55}
|
||||
textAlign="center"
|
||||
>
|
||||
{enabled ? (playing ? "♪ NOW PLAYING" : "SOUNDCLAW") : "NOT INSTALLED"}
|
||||
</Text>
|
||||
</Billboard>
|
||||
|
||||
{/* Speaker grill (replaces record slot). */}
|
||||
<mesh position={[0, 0.7, 0.31]}>
|
||||
<planeGeometry args={[0.52, 0.38]} />
|
||||
<meshStandardMaterial color="#042f2e" roughness={0.9} metalness={0.1} />
|
||||
</mesh>
|
||||
{/* Horizontal grill lines. */}
|
||||
{[-0.14, -0.07, 0, 0.07, 0.14].map((y) => (
|
||||
<mesh key={y} position={[0, 0.7 + y, 0.315]}>
|
||||
<boxGeometry args={[0.48, 0.01, 0.005]} />
|
||||
<meshStandardMaterial color={C.metalDark} metalness={0.6} roughness={0.4} />
|
||||
</mesh>
|
||||
))}
|
||||
|
||||
{/* Spinning vinyl disc (small, subtle). */}
|
||||
<mesh
|
||||
ref={recordRef}
|
||||
position={[0, 0.75, 0.315]}
|
||||
rotation={[Math.PI / 2, 0, 0]}
|
||||
>
|
||||
<cylinderGeometry args={[0.1, 0.1, 0.008, 32]} />
|
||||
<meshStandardMaterial color="#0a0a0a" roughness={0.6} metalness={0.3} />
|
||||
</mesh>
|
||||
{/* Record label. */}
|
||||
<mesh position={[0, 0.75, 0.32]} rotation={[Math.PI / 2, 0, 0]}>
|
||||
<circleGeometry args={[0.04, 32]} />
|
||||
<meshStandardMaterial color={C.recordLabel} emissive={C.neon} emissiveIntensity={playing ? 0.8 : 0.3} />
|
||||
</mesh>
|
||||
|
||||
{/* Coloured selection buttons (grey when disabled). */}
|
||||
<group position={[0, 0.5, 0.31]}>
|
||||
{BUTTON_COLORS.map((color, i) => (
|
||||
<mesh key={i} position={[-0.15 + i * 0.075, 0, 0.01]}>
|
||||
<cylinderGeometry args={[0.025, 0.025, 0.02, 16]} />
|
||||
<meshStandardMaterial
|
||||
color={enabled ? color : "#555"}
|
||||
emissive={enabled ? color : "#222"}
|
||||
emissiveIntensity={enabled ? 0.5 : 0.05}
|
||||
/>
|
||||
</mesh>
|
||||
))}
|
||||
</group>
|
||||
|
||||
{/* Side grilles (translucent). */}
|
||||
<mesh position={[-0.35, 0.75, 0]} rotation={[0, Math.PI / 2, 0]}>
|
||||
<planeGeometry args={[0.8, 0.6]} />
|
||||
<meshStandardMaterial color={tint(C.metalDark, "#3a3a3a")} roughness={0.5} metalness={0.4} transparent opacity={0.8} />
|
||||
</mesh>
|
||||
<mesh position={[0.35, 0.75, 0]} rotation={[0, -Math.PI / 2, 0]}>
|
||||
<planeGeometry args={[0.8, 0.6]} />
|
||||
<meshStandardMaterial color={tint(C.metalDark, "#3a3a3a")} roughness={0.5} metalness={0.4} transparent opacity={0.8} />
|
||||
</mesh>
|
||||
|
||||
{/* Base plinth. */}
|
||||
<mesh position={[0, 0.05, 0]} receiveShadow>
|
||||
<boxGeometry args={[0.9, 0.1, 0.7]} />
|
||||
<meshStandardMaterial color={tint(C.cabinetDark, "#2a2a2a")} roughness={0.7} metalness={0.1} />
|
||||
</mesh>
|
||||
|
||||
{/* Floating "Install skill" hint above the machine when disabled and hovered. */}
|
||||
{!enabled && (localHovered || isHovered) && (
|
||||
<Billboard position={[0, 2.0, 0]} follow={false}>
|
||||
<Text fontSize={0.07} color="#facc15" anchorX="center" anchorY="middle" outlineWidth={0.01} outlineColor="#000">
|
||||
Click to install SOUNDCLAW
|
||||
</Text>
|
||||
</Billboard>
|
||||
)}
|
||||
|
||||
{/* Green point light when a song is playing. */}
|
||||
{playing && (
|
||||
<pointLight
|
||||
ref={glowRef}
|
||||
position={[0, 1.2, 0.5]}
|
||||
color={C.neonActive}
|
||||
intensity={1}
|
||||
distance={3}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Green hover indicator dot above the machine. */}
|
||||
{(localHovered || isHovered) && (
|
||||
<mesh position={[0, 1.68, 0]}>
|
||||
<sphereGeometry args={[0.05, 16, 16]} />
|
||||
<meshStandardMaterial color="#00FF00" emissive="#00FF00" emissiveIntensity={1} />
|
||||
</mesh>
|
||||
)}
|
||||
|
||||
{/* Selection highlight ring when selected. */}
|
||||
{isSelected && (
|
||||
<mesh position={[0, 0.75, 0]}>
|
||||
<torusGeometry args={[0.52, 0.03, 12, 48]} />
|
||||
<meshStandardMaterial color="#fbbf24" emissive="#fbbf24" emissiveIntensity={1} />
|
||||
</mesh>
|
||||
)}
|
||||
</group>
|
||||
</group>
|
||||
);
|
||||
}
|
||||
@@ -8,7 +8,10 @@ import {
|
||||
WALK_ANIM_SPEED,
|
||||
} from "@/features/retro-office/core/constants";
|
||||
import { toWorld } from "@/features/retro-office/core/geometry";
|
||||
import type { JanitorActor, RenderAgent } from "@/features/retro-office/core/types";
|
||||
import type {
|
||||
JanitorActor,
|
||||
RenderAgent,
|
||||
} from "@/features/retro-office/core/types";
|
||||
import { AgentModelProps } from "@/features/retro-office/objects/types";
|
||||
|
||||
export const AgentModel = memo(function AgentModel({
|
||||
@@ -57,7 +60,7 @@ export const AgentModel = memo(function AgentModel({
|
||||
const pos = useRef(new THREE.Vector3(0, 0, 0));
|
||||
const resolvedAppearance = useMemo(
|
||||
() => appearance ?? createDefaultAgentAvatarProfile(agentId),
|
||||
[agentId, appearance]
|
||||
[agentId, appearance],
|
||||
);
|
||||
|
||||
useFrame(() => {
|
||||
@@ -76,6 +79,7 @@ export const AgentModel = memo(function AgentModel({
|
||||
while (rotDelta < -Math.PI) rotDelta += Math.PI * 2;
|
||||
groupRef.current.rotation.y += rotDelta * 0.12;
|
||||
const isWorkout = agent.state === "working_out";
|
||||
const isDancing = agent.state === "dancing";
|
||||
const isJanitor = "role" in agent && agent.role === "janitor";
|
||||
const janitorTool = isJanitor
|
||||
? (agent as RenderAgent & JanitorActor).janitorTool
|
||||
@@ -83,12 +87,18 @@ export const AgentModel = memo(function AgentModel({
|
||||
const workoutStyle = agent.workoutStyle ?? "lift";
|
||||
const frameValue = agent.frame + (agent.phaseOffset ?? 0) / WALK_ANIM_SPEED;
|
||||
const walkPhase = Math.sin(frameValue * WALK_ANIM_SPEED);
|
||||
const workoutPhase = Math.sin(agent.frame * 0.18 + (agent.phaseOffset ?? 0));
|
||||
const workoutPushPhase = Math.sin(agent.frame * 0.18 + (agent.phaseOffset ?? 0) + Math.PI / 2);
|
||||
const workoutPhase = Math.sin(
|
||||
agent.frame * 0.18 + (agent.phaseOffset ?? 0),
|
||||
);
|
||||
const workoutPushPhase = Math.sin(
|
||||
agent.frame * 0.18 + (agent.phaseOffset ?? 0) + Math.PI / 2,
|
||||
);
|
||||
groupRef.current.rotation.z = 0;
|
||||
groupRef.current.rotation.x =
|
||||
agent.state === "sitting"
|
||||
? -0.15
|
||||
: isDancing
|
||||
? Math.sin(agent.frame * 0.18 + (agent.phaseOffset ?? 0)) * 0.06
|
||||
: isWorkout
|
||||
? workoutStyle === "bike"
|
||||
? 0.18
|
||||
@@ -107,6 +117,8 @@ export const AgentModel = memo(function AgentModel({
|
||||
const bounce =
|
||||
agent.state === "walking"
|
||||
? Math.sin(frameValue * WALK_ANIM_SPEED) * 0.04
|
||||
: isDancing
|
||||
? 0.03 + Math.abs(Math.sin(agent.frame * 0.22 + (agent.phaseOffset ?? 0))) * 0.05
|
||||
: isWorkout
|
||||
? workoutStyle === "stretch"
|
||||
? 0.012 + Math.abs(workoutPhase) * 0.018
|
||||
@@ -129,6 +141,11 @@ export const AgentModel = memo(function AgentModel({
|
||||
leftArmRef.current.rotation.z = -0.08;
|
||||
} else if (agent.state === "walking") {
|
||||
leftArmRef.current.rotation.x = walkPhase * 0.4;
|
||||
} else if (isDancing) {
|
||||
leftArmRef.current.rotation.x = -0.8 + Math.sin(agent.frame * 0.22) * 0.9;
|
||||
leftArmRef.current.rotation.z = -0.45 + Math.cos(agent.frame * 0.16) * 0.18;
|
||||
leftArmRef.current.rotation.y = -0.08;
|
||||
groupRef.current.rotation.z = Math.sin(agent.frame * 0.12) * 0.08;
|
||||
} else if (isWorkout) {
|
||||
if (workoutStyle === "run") {
|
||||
leftArmRef.current.rotation.x = -(0.28 + workoutPhase * 1.05);
|
||||
@@ -138,11 +155,17 @@ export const AgentModel = memo(function AgentModel({
|
||||
leftArmRef.current.rotation.z = -0.18;
|
||||
leftArmRef.current.rotation.y = -0.12;
|
||||
} else if (workoutStyle === "row") {
|
||||
leftArmRef.current.rotation.x = -(0.95 - Math.max(0, workoutPhase) * 0.7);
|
||||
leftArmRef.current.rotation.x = -(
|
||||
0.95 -
|
||||
Math.max(0, workoutPhase) * 0.7
|
||||
);
|
||||
leftArmRef.current.rotation.z = -0.16;
|
||||
leftArmRef.current.rotation.y = -0.1;
|
||||
} else if (workoutStyle === "box") {
|
||||
leftArmRef.current.rotation.x = -(0.92 + Math.max(0, workoutPushPhase) * 0.45);
|
||||
leftArmRef.current.rotation.x = -(
|
||||
0.92 +
|
||||
Math.max(0, workoutPushPhase) * 0.45
|
||||
);
|
||||
leftArmRef.current.rotation.z = -0.52;
|
||||
leftArmRef.current.rotation.y = -0.06;
|
||||
groupRef.current.rotation.z = 0.05;
|
||||
@@ -151,12 +174,16 @@ export const AgentModel = memo(function AgentModel({
|
||||
leftArmRef.current.rotation.z = -0.42;
|
||||
leftArmRef.current.rotation.y = -0.08;
|
||||
} else {
|
||||
leftArmRef.current.rotation.x = -(0.28 + Math.abs(workoutPhase) * 0.28);
|
||||
leftArmRef.current.rotation.x = -(
|
||||
0.28 +
|
||||
Math.abs(workoutPhase) * 0.28
|
||||
);
|
||||
leftArmRef.current.rotation.z = -0.58;
|
||||
leftArmRef.current.rotation.y = -0.12;
|
||||
}
|
||||
} else if (agent.pingPongUntil) {
|
||||
leftArmRef.current.rotation.x = 0.2 + Math.sin(agent.frame * 0.08) * 0.28;
|
||||
leftArmRef.current.rotation.x =
|
||||
0.2 + Math.sin(agent.frame * 0.08) * 0.28;
|
||||
} else if (agent.state === "sitting") {
|
||||
leftArmRef.current.rotation.x = 0.3;
|
||||
}
|
||||
@@ -171,6 +198,11 @@ export const AgentModel = memo(function AgentModel({
|
||||
rightArmRef.current.rotation.z = 0.08;
|
||||
} else if (agent.state === "walking") {
|
||||
rightArmRef.current.rotation.x = -walkPhase * 0.4;
|
||||
} else if (isDancing) {
|
||||
rightArmRef.current.rotation.x = -0.8 - Math.sin(agent.frame * 0.22) * 0.9;
|
||||
rightArmRef.current.rotation.z = 0.45 - Math.cos(agent.frame * 0.16) * 0.18;
|
||||
rightArmRef.current.rotation.y = 0.08;
|
||||
groupRef.current.rotation.z = Math.sin(agent.frame * 0.12) * 0.08;
|
||||
} else if (isWorkout) {
|
||||
if (workoutStyle === "run") {
|
||||
rightArmRef.current.rotation.x = -(0.28 - workoutPhase * 1.05);
|
||||
@@ -180,11 +212,17 @@ export const AgentModel = memo(function AgentModel({
|
||||
rightArmRef.current.rotation.z = 0.18;
|
||||
rightArmRef.current.rotation.y = 0.12;
|
||||
} else if (workoutStyle === "row") {
|
||||
rightArmRef.current.rotation.x = -(0.95 - Math.max(0, -workoutPhase) * 0.7);
|
||||
rightArmRef.current.rotation.x = -(
|
||||
0.95 -
|
||||
Math.max(0, -workoutPhase) * 0.7
|
||||
);
|
||||
rightArmRef.current.rotation.z = 0.16;
|
||||
rightArmRef.current.rotation.y = 0.1;
|
||||
} else if (workoutStyle === "box") {
|
||||
rightArmRef.current.rotation.x = -(0.92 + Math.max(0, -workoutPushPhase) * 0.45);
|
||||
rightArmRef.current.rotation.x = -(
|
||||
0.92 +
|
||||
Math.max(0, -workoutPushPhase) * 0.45
|
||||
);
|
||||
rightArmRef.current.rotation.z = 0.52;
|
||||
rightArmRef.current.rotation.y = 0.06;
|
||||
groupRef.current.rotation.z = -0.05;
|
||||
@@ -193,12 +231,16 @@ export const AgentModel = memo(function AgentModel({
|
||||
rightArmRef.current.rotation.z = 0.42;
|
||||
rightArmRef.current.rotation.y = 0.08;
|
||||
} else {
|
||||
rightArmRef.current.rotation.x = -(0.28 + Math.abs(workoutPhase) * 0.28);
|
||||
rightArmRef.current.rotation.x = -(
|
||||
0.28 +
|
||||
Math.abs(workoutPhase) * 0.28
|
||||
);
|
||||
rightArmRef.current.rotation.z = 0.58;
|
||||
rightArmRef.current.rotation.y = 0.12;
|
||||
}
|
||||
} else if (agent.pingPongUntil) {
|
||||
rightArmRef.current.rotation.x = 0.08 - Math.sin(agent.frame * 0.08) * 0.16;
|
||||
rightArmRef.current.rotation.x =
|
||||
0.08 - Math.sin(agent.frame * 0.08) * 0.16;
|
||||
} else if (agent.state === "sitting") {
|
||||
rightArmRef.current.rotation.x = 0.3;
|
||||
}
|
||||
@@ -207,6 +249,8 @@ export const AgentModel = memo(function AgentModel({
|
||||
leftLegRef.current.rotation.x =
|
||||
agent.state === "walking"
|
||||
? walkPhase * 0.35
|
||||
: isDancing
|
||||
? Math.sin(agent.frame * 0.22 + (agent.phaseOffset ?? 0)) * 0.35
|
||||
: isWorkout
|
||||
? workoutStyle === "run"
|
||||
? workoutPhase * 0.7
|
||||
@@ -225,6 +269,8 @@ export const AgentModel = memo(function AgentModel({
|
||||
rightLegRef.current.rotation.x =
|
||||
agent.state === "walking"
|
||||
? -walkPhase * 0.35
|
||||
: isDancing
|
||||
? -Math.sin(agent.frame * 0.22 + (agent.phaseOffset ?? 0)) * 0.35
|
||||
: isWorkout
|
||||
? workoutStyle === "run"
|
||||
? -workoutPhase * 0.7
|
||||
@@ -241,7 +287,7 @@ export const AgentModel = memo(function AgentModel({
|
||||
}
|
||||
|
||||
const working =
|
||||
agent.state === "sitting" || isWorkout || agent.status === "working";
|
||||
agent.state === "sitting" || isWorkout || isDancing || agent.status === "working";
|
||||
const isError = agent.status === "error";
|
||||
const isAway = agent.state === "away";
|
||||
|
||||
@@ -343,7 +389,8 @@ export const AgentModel = memo(function AgentModel({
|
||||
const showFrownCorners = isError;
|
||||
if (leftMouthCornerRef.current && rightMouthCornerRef.current) {
|
||||
leftMouthCornerRef.current.visible = showSmileCorners || showFrownCorners;
|
||||
rightMouthCornerRef.current.visible = showSmileCorners || showFrownCorners;
|
||||
rightMouthCornerRef.current.visible =
|
||||
showSmileCorners || showFrownCorners;
|
||||
leftMouthCornerRef.current.position.set(-0.031, 0.434, 0.074);
|
||||
rightMouthCornerRef.current.position.set(0.031, 0.434, 0.074);
|
||||
if (showFrownCorners) {
|
||||
@@ -395,7 +442,8 @@ export const AgentModel = memo(function AgentModel({
|
||||
|
||||
if (speechBubbleRef.current) {
|
||||
const bubbleVisible =
|
||||
!suppressSpeechBubble && (showSpeech || bumpTalking || ambientBubbleVisible);
|
||||
!suppressSpeechBubble &&
|
||||
(showSpeech || bumpTalking || ambientBubbleVisible);
|
||||
speechBubbleRef.current.visible = bubbleVisible;
|
||||
if (bubbleVisible) {
|
||||
if (showSpeech && speechText?.trim()) {
|
||||
@@ -440,8 +488,13 @@ export const AgentModel = memo(function AgentModel({
|
||||
const showBroom = isJanitor && janitorTool === "broom";
|
||||
heldCleaningToolRef.current.visible = showBroom;
|
||||
if (showBroom) {
|
||||
const sweep = agent.state === "walking" ? Math.sin(agent.frame * 0.08) * 0.08 : 0;
|
||||
heldCleaningToolRef.current.position.set(-0.02, -0.2, 0.08 + sweep * 0.06);
|
||||
const sweep =
|
||||
agent.state === "walking" ? Math.sin(agent.frame * 0.08) * 0.08 : 0;
|
||||
heldCleaningToolRef.current.position.set(
|
||||
-0.02,
|
||||
-0.2,
|
||||
0.08 + sweep * 0.06,
|
||||
);
|
||||
heldCleaningToolRef.current.rotation.set(-0.8, 0.18, -0.18);
|
||||
}
|
||||
}
|
||||
@@ -712,7 +765,11 @@ export const AgentModel = memo(function AgentModel({
|
||||
<boxGeometry args={[0.05, 0.05, 0.05]} />
|
||||
<meshLambertMaterial color={skin} />
|
||||
</mesh>
|
||||
<group ref={heldPaddleRef} position={[-0.01, -0.21, 0.07]} visible={false}>
|
||||
<group
|
||||
ref={heldPaddleRef}
|
||||
position={[-0.01, -0.21, 0.07]}
|
||||
visible={false}
|
||||
>
|
||||
<mesh rotation={[-Math.PI / 2, 0, 0]}>
|
||||
<cylinderGeometry args={[0.042, 0.042, 0.012, 18]} />
|
||||
<meshStandardMaterial
|
||||
@@ -746,7 +803,11 @@ export const AgentModel = memo(function AgentModel({
|
||||
</mesh>
|
||||
</group>
|
||||
{/* Vacuum cleaner: larger upright silhouette so it reads clearly in-scene. */}
|
||||
<group ref={heldBucketRef} position={[-0.08, -0.1, 0.18]} visible={false}>
|
||||
<group
|
||||
ref={heldBucketRef}
|
||||
position={[-0.08, -0.1, 0.18]}
|
||||
visible={false}
|
||||
>
|
||||
<mesh position={[0, -0.02, 0]}>
|
||||
<boxGeometry args={[0.015, 0.3, 0.015]} />
|
||||
<meshStandardMaterial color="#555" roughness={0.72} />
|
||||
@@ -761,11 +822,19 @@ export const AgentModel = memo(function AgentModel({
|
||||
</mesh>
|
||||
<mesh position={[0.02, -0.11, 0.035]} rotation={[0, Math.PI / 2, 0]}>
|
||||
<torusGeometry args={[0.03, 0.005, 10, 18, Math.PI]} />
|
||||
<meshStandardMaterial color="#94a3b8" roughness={0.36} metalness={0.18} />
|
||||
<meshStandardMaterial
|
||||
color="#94a3b8"
|
||||
roughness={0.36}
|
||||
metalness={0.18}
|
||||
/>
|
||||
</mesh>
|
||||
</group>
|
||||
{/* Floor scrubber: prominent handle, body, and wide cleaning base. */}
|
||||
<group ref={heldScrubberRef} position={[-0.1, -0.08, 0.2]} visible={false}>
|
||||
<group
|
||||
ref={heldScrubberRef}
|
||||
position={[-0.1, -0.08, 0.2]}
|
||||
visible={false}
|
||||
>
|
||||
<mesh position={[0, -0.02, 0]}>
|
||||
<boxGeometry args={[0.015, 0.32, 0.015]} />
|
||||
<meshStandardMaterial color="#777" roughness={0.7} />
|
||||
@@ -945,11 +1014,19 @@ export const AgentModel = memo(function AgentModel({
|
||||
<boxGeometry args={[0.05, 0.014, 0.01]} />
|
||||
<meshBasicMaterial color="#9c4a4a" />
|
||||
</mesh>
|
||||
<mesh ref={leftMouthCornerRef} position={[-0.031, 0.438, 0.074]} visible={false}>
|
||||
<mesh
|
||||
ref={leftMouthCornerRef}
|
||||
position={[-0.031, 0.438, 0.074]}
|
||||
visible={false}
|
||||
>
|
||||
<boxGeometry args={[0.014, 0.014, 0.01]} />
|
||||
<meshBasicMaterial color="#9c4a4a" />
|
||||
</mesh>
|
||||
<mesh ref={rightMouthCornerRef} position={[0.031, 0.438, 0.074]} visible={false}>
|
||||
<mesh
|
||||
ref={rightMouthCornerRef}
|
||||
position={[0.031, 0.438, 0.074]}
|
||||
visible={false}
|
||||
>
|
||||
<boxGeometry args={[0.014, 0.014, 0.01]} />
|
||||
<meshBasicMaterial color="#9c4a4a" />
|
||||
</mesh>
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
"use client";
|
||||
|
||||
import { useJukeboxStore } from "./store";
|
||||
import { searchTracks } from "./spotifyApi";
|
||||
|
||||
type BrowserJukeboxCommand =
|
||||
| { kind: "pause" }
|
||||
| { kind: "resume" }
|
||||
| { kind: "next" }
|
||||
| { kind: "previous" }
|
||||
| { kind: "play"; query: string | null };
|
||||
|
||||
export type BrowserJukeboxExecutionResult =
|
||||
| { ok: false }
|
||||
| { ok: true; reply: string };
|
||||
|
||||
const normalize = (value: string): string =>
|
||||
value.trim().toLowerCase().replace(/\s+/g, " ");
|
||||
|
||||
const stripPlayPrefix = (normalizedMessage: string): string => {
|
||||
let value = normalizedMessage;
|
||||
value = value.replace(/\b(play|queue|put on|start)\b/g, " ");
|
||||
value = value.replace(/\b(on spotify|from spotify|on the jukebox|with the jukebox)\b/g, " ");
|
||||
value = value.replace(/\b(the )?(jukebox|spotify|music|song|songs|track|tracks)\b/g, " ");
|
||||
value = value.replace(/\s+/g, " ").trim();
|
||||
return value;
|
||||
};
|
||||
|
||||
export const parseBrowserJukeboxCommand = (
|
||||
message: string | null | undefined,
|
||||
): BrowserJukeboxCommand | null => {
|
||||
const normalized = normalize(message ?? "");
|
||||
if (!normalized) return null;
|
||||
|
||||
if (/\b(pause|stop music|stop playback|stop song)\b/.test(normalized)) {
|
||||
return { kind: "pause" };
|
||||
}
|
||||
if (/\b(resume|continue|unpause)\b/.test(normalized)) {
|
||||
return { kind: "resume" };
|
||||
}
|
||||
if (/\b(next|skip)\b/.test(normalized)) {
|
||||
return { kind: "next" };
|
||||
}
|
||||
if (/\b(previous|prev|back)\b/.test(normalized)) {
|
||||
return { kind: "previous" };
|
||||
}
|
||||
if (/\b(play|queue|put on|start)\b/.test(normalized)) {
|
||||
const query = stripPlayPrefix(normalized);
|
||||
return { kind: "play", query: query.length > 0 ? query : null };
|
||||
}
|
||||
|
||||
return null;
|
||||
};
|
||||
|
||||
export const executeBrowserJukeboxCommand = async (
|
||||
message: string | null | undefined,
|
||||
): Promise<BrowserJukeboxExecutionResult> => {
|
||||
const command = parseBrowserJukeboxCommand(message);
|
||||
if (!command) return { ok: false };
|
||||
|
||||
const store = useJukeboxStore.getState();
|
||||
store.init();
|
||||
const { token } = useJukeboxStore.getState();
|
||||
if (!token) return { ok: false };
|
||||
|
||||
switch (command.kind) {
|
||||
case "pause":
|
||||
await useJukeboxStore.getState().pause();
|
||||
return { ok: true, reply: "Paused the office jukebox." };
|
||||
case "resume":
|
||||
await useJukeboxStore.getState().resume();
|
||||
return { ok: true, reply: "Resumed the office jukebox." };
|
||||
case "next":
|
||||
await useJukeboxStore.getState().next();
|
||||
return { ok: true, reply: "Skipped to the next track on the office jukebox." };
|
||||
case "previous":
|
||||
await useJukeboxStore.getState().previous();
|
||||
return { ok: true, reply: "Went back to the previous track on the office jukebox." };
|
||||
case "play": {
|
||||
if (!command.query) {
|
||||
await useJukeboxStore.getState().resume();
|
||||
return { ok: true, reply: "Started the office jukebox." };
|
||||
}
|
||||
const results = await searchTracks(token, command.query);
|
||||
useJukeboxStore.setState({
|
||||
searchQuery: command.query,
|
||||
searchResults: results,
|
||||
});
|
||||
if (results.length === 0) {
|
||||
return { ok: false };
|
||||
}
|
||||
await useJukeboxStore.getState().play(results[0].uri);
|
||||
const top = results[0];
|
||||
const artist = top.artists[0]?.name ?? "Unknown artist";
|
||||
return {
|
||||
ok: true,
|
||||
reply: `Playing ${artist} - "${top.name}" on the office jukebox.`,
|
||||
};
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,188 @@
|
||||
"use client";
|
||||
|
||||
// Spotify PKCE OAuth helpers.
|
||||
// No client secret is needed; PKCE is the recommended flow for SPAs.
|
||||
|
||||
const STORAGE_PREFIX = "soundclaw_";
|
||||
const TOKEN_KEY = `${STORAGE_PREFIX}token`;
|
||||
const EXPIRY_KEY = `${STORAGE_PREFIX}expiry`;
|
||||
const VERIFIER_KEY = `${STORAGE_PREFIX}verifier`;
|
||||
const CLIENT_ID_KEY = `${STORAGE_PREFIX}client_id`;
|
||||
const CALLBACK_BASE_URL_KEY = `${STORAGE_PREFIX}callback_base_url`;
|
||||
const REDIRECT_URI_KEY = `${STORAGE_PREFIX}redirect_uri`;
|
||||
const STATE_KEY = `${STORAGE_PREFIX}state`;
|
||||
|
||||
export const SPOTIFY_SCOPES = [
|
||||
"user-read-playback-state",
|
||||
"user-modify-playback-state",
|
||||
"user-read-currently-playing",
|
||||
"streaming",
|
||||
"playlist-read-private",
|
||||
"playlist-read-collaborative",
|
||||
].join(" ");
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Client ID persistence
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const saveClientId = (id: string) => {
|
||||
try { localStorage.setItem(CLIENT_ID_KEY, id); } catch { /* ignore */ }
|
||||
};
|
||||
|
||||
export const loadClientId = (): string => {
|
||||
try { return localStorage.getItem(CLIENT_ID_KEY) ?? ""; } catch { return ""; }
|
||||
};
|
||||
|
||||
const normalizeBaseUrl = (value: string): string => value.trim().replace(/\/+$/, "");
|
||||
|
||||
export const saveCallbackBaseUrl = (url: string) => {
|
||||
try { localStorage.setItem(CALLBACK_BASE_URL_KEY, normalizeBaseUrl(url)); } catch { /* ignore */ }
|
||||
};
|
||||
|
||||
export const loadCallbackBaseUrl = (): string => {
|
||||
try { return localStorage.getItem(CALLBACK_BASE_URL_KEY) ?? ""; } catch { return ""; }
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Token persistence
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const saveToken = (token: string, expiresInSeconds: number) => {
|
||||
try {
|
||||
localStorage.setItem(TOKEN_KEY, token);
|
||||
localStorage.setItem(EXPIRY_KEY, String(Date.now() + expiresInSeconds * 1000));
|
||||
} catch { /* ignore */ }
|
||||
};
|
||||
|
||||
export const loadToken = (): string | null => {
|
||||
try {
|
||||
const token = localStorage.getItem(TOKEN_KEY);
|
||||
const expiry = Number(localStorage.getItem(EXPIRY_KEY) ?? "0");
|
||||
if (!token || Date.now() > expiry) return null;
|
||||
return token;
|
||||
} catch { return null; }
|
||||
};
|
||||
|
||||
export const clearToken = () => {
|
||||
try {
|
||||
localStorage.removeItem(TOKEN_KEY);
|
||||
localStorage.removeItem(EXPIRY_KEY);
|
||||
localStorage.removeItem(VERIFIER_KEY);
|
||||
localStorage.removeItem(REDIRECT_URI_KEY);
|
||||
localStorage.removeItem(STATE_KEY);
|
||||
} catch { /* ignore */ }
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// PKCE helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const generateRandom = (length: number): string => {
|
||||
const chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~";
|
||||
const array = new Uint8Array(length);
|
||||
crypto.getRandomValues(array);
|
||||
return Array.from(array, (b) => chars[b % chars.length]).join("");
|
||||
};
|
||||
|
||||
const sha256 = async (plain: string): Promise<ArrayBuffer> => {
|
||||
const encoder = new TextEncoder();
|
||||
const data = encoder.encode(plain);
|
||||
return crypto.subtle.digest("SHA-256", data);
|
||||
};
|
||||
|
||||
const base64urlEncode = (buffer: ArrayBuffer): string =>
|
||||
btoa(String.fromCharCode(...new Uint8Array(buffer)))
|
||||
.replace(/\+/g, "-")
|
||||
.replace(/\//g, "_")
|
||||
.replace(/=+$/, "");
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// OAuth redirect.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const startSpotifyAuth = async (
|
||||
clientId: string,
|
||||
redirectUri: string,
|
||||
popup?: Window | null,
|
||||
) => {
|
||||
const verifier = generateRandom(64);
|
||||
const state = generateRandom(32);
|
||||
const challenge = base64urlEncode(await sha256(verifier));
|
||||
try {
|
||||
localStorage.setItem(VERIFIER_KEY, verifier);
|
||||
localStorage.setItem(REDIRECT_URI_KEY, redirectUri);
|
||||
localStorage.setItem(STATE_KEY, state);
|
||||
} catch { /* ignore */ }
|
||||
|
||||
const params = new URLSearchParams({
|
||||
client_id: clientId,
|
||||
response_type: "code",
|
||||
redirect_uri: redirectUri,
|
||||
state,
|
||||
code_challenge_method: "S256",
|
||||
code_challenge: challenge,
|
||||
scope: SPOTIFY_SCOPES,
|
||||
});
|
||||
|
||||
const authorizeUrl = `https://accounts.spotify.com/authorize?${params}`;
|
||||
if (popup && !popup.closed) {
|
||||
popup.location.href = authorizeUrl;
|
||||
popup.focus();
|
||||
return;
|
||||
}
|
||||
window.location.href = authorizeUrl;
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Token exchange (called after redirect back with ?code=...)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const exchangeCodeForToken = async (
|
||||
code: string,
|
||||
clientId: string,
|
||||
redirectUri?: string,
|
||||
): Promise<boolean> => {
|
||||
try {
|
||||
const verifier = localStorage.getItem(VERIFIER_KEY);
|
||||
const resolvedRedirectUri = redirectUri ?? localStorage.getItem(REDIRECT_URI_KEY);
|
||||
if (!verifier || !resolvedRedirectUri) return false;
|
||||
|
||||
const body = new URLSearchParams({
|
||||
client_id: clientId,
|
||||
grant_type: "authorization_code",
|
||||
code,
|
||||
redirect_uri: resolvedRedirectUri,
|
||||
code_verifier: verifier,
|
||||
});
|
||||
|
||||
const res = await fetch("https://accounts.spotify.com/api/token", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/x-www-form-urlencoded" },
|
||||
body,
|
||||
});
|
||||
|
||||
if (!res.ok) return false;
|
||||
|
||||
const json = await res.json() as { access_token: string; expires_in: number };
|
||||
saveToken(json.access_token, json.expires_in);
|
||||
localStorage.removeItem(VERIFIER_KEY);
|
||||
localStorage.removeItem(REDIRECT_URI_KEY);
|
||||
localStorage.removeItem(STATE_KEY);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Redirect URI helper.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const buildRedirectUri = (callbackBaseUrl?: string): string => {
|
||||
const baseUrl = normalizeBaseUrl(callbackBaseUrl ?? loadCallbackBaseUrl());
|
||||
return baseUrl ? `${baseUrl}/spotify/callback` : "";
|
||||
};
|
||||
|
||||
export const loadAuthState = (): string => {
|
||||
try { return localStorage.getItem(STATE_KEY) ?? ""; } catch { return ""; }
|
||||
};
|
||||
@@ -0,0 +1,45 @@
|
||||
"use client";
|
||||
|
||||
type JukeboxDisabledPanelProps = {
|
||||
onClose: () => void;
|
||||
onInstall: () => void;
|
||||
};
|
||||
|
||||
export function JukeboxDisabledPanel({ onClose, onInstall }: JukeboxDisabledPanelProps) {
|
||||
return (
|
||||
<div className="absolute inset-0 z-50 flex items-center justify-center bg-black/60 p-6">
|
||||
<div className="w-full max-w-sm rounded-3xl border border-slate-700/40 bg-slate-950/95 p-8 text-center shadow-2xl">
|
||||
{/* Jukebox icon. */}
|
||||
<div className="mx-auto mb-4 flex h-16 w-16 items-center justify-center rounded-2xl border border-slate-700/40 bg-slate-800/60 text-4xl">
|
||||
🎵
|
||||
</div>
|
||||
|
||||
<div className="font-mono text-[10px] uppercase tracking-[0.24em] text-slate-500">
|
||||
Soundclaw
|
||||
</div>
|
||||
<h2 className="mt-1 text-xl font-semibold text-white">Jukebox Not Installed</h2>
|
||||
<p className="mt-3 text-sm leading-relaxed text-slate-400">
|
||||
Install the <span className="text-cyan-400">SOUNDCLAW</span> skill to let your agents
|
||||
pick and play music right from the office jukebox.
|
||||
</p>
|
||||
|
||||
<div className="mt-6 flex flex-col gap-3">
|
||||
<button
|
||||
type="button"
|
||||
className="rounded-xl bg-cyan-500 px-5 py-2.5 text-sm font-medium text-slate-950 transition hover:bg-cyan-400 active:scale-95"
|
||||
onClick={onInstall}
|
||||
>
|
||||
Install SOUNDCLAW skill
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="rounded-xl border border-slate-700/40 px-5 py-2.5 text-sm text-slate-400 transition hover:bg-slate-800/50"
|
||||
onClick={onClose}
|
||||
>
|
||||
Dismiss
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,463 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import { useJukeboxStore } from "../store";
|
||||
import {
|
||||
startSpotifyAuth,
|
||||
buildRedirectUri,
|
||||
loadToken,
|
||||
exchangeCodeForToken,
|
||||
loadCallbackBaseUrl,
|
||||
saveCallbackBaseUrl,
|
||||
loadAuthState,
|
||||
} from "../auth";
|
||||
import type { SpotifyTrack } from "../spotifyApi";
|
||||
|
||||
type JukeboxPanelProps = {
|
||||
onClose: () => void;
|
||||
selectedAgentName?: string | null;
|
||||
client?: unknown;
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Root panel
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function JukeboxPanel({ onClose }: JukeboxPanelProps) {
|
||||
const { view, init } = useJukeboxStore();
|
||||
|
||||
useEffect(() => {
|
||||
init();
|
||||
const handleMessage = (event: MessageEvent) => {
|
||||
const callbackBaseUrl = loadCallbackBaseUrl();
|
||||
if (!callbackBaseUrl) return;
|
||||
const callbackOrigin = new URL(callbackBaseUrl).origin;
|
||||
if (event.origin !== callbackOrigin) return;
|
||||
const payload = event.data as
|
||||
| {
|
||||
type?: string;
|
||||
code?: string;
|
||||
error?: string;
|
||||
state?: string;
|
||||
}
|
||||
| undefined;
|
||||
if (!payload || payload.type !== "soundclaw-spotify-auth") return;
|
||||
if (payload.error) return;
|
||||
if (!payload.code) return;
|
||||
if (payload.state !== loadAuthState()) return;
|
||||
|
||||
const { clientId, setToken } = useJukeboxStore.getState();
|
||||
void exchangeCodeForToken(payload.code, clientId, buildRedirectUri(callbackBaseUrl)).then(
|
||||
(ok) => {
|
||||
if (!ok) return;
|
||||
const token = loadToken();
|
||||
if (token) setToken(token);
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
window.addEventListener("message", handleMessage);
|
||||
return () => window.removeEventListener("message", handleMessage);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<div className="absolute inset-0 z-50 flex items-center justify-center bg-black/70 p-6 backdrop-blur-sm">
|
||||
<div
|
||||
className="w-full max-w-2xl overflow-hidden rounded-3xl border border-cyan-500/20 bg-slate-950/98 shadow-2xl"
|
||||
style={{ maxHeight: "90vh" }}
|
||||
>
|
||||
{/* Header. */}
|
||||
<div className="flex items-center justify-between border-b border-white/5 px-6 py-4">
|
||||
<div className="flex items-center gap-3">
|
||||
<span className="text-2xl">🎵</span>
|
||||
<div>
|
||||
<div className="font-mono text-[10px] uppercase tracking-[0.2em] text-cyan-400/70">
|
||||
Soundclaw
|
||||
</div>
|
||||
<h2 className="text-base font-semibold text-white">Office Jukebox</h2>
|
||||
</div>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClose}
|
||||
className="rounded-full border border-white/10 px-4 py-1.5 text-sm text-slate-400 transition hover:bg-white/5 hover:text-white"
|
||||
>
|
||||
Close
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Body. */}
|
||||
<div className="overflow-y-auto" style={{ maxHeight: "calc(90vh - 68px)" }}>
|
||||
{view === "setup" ? <SetupView /> : <PlayerView />}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Setup view — shown before the user authenticates
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function SetupView() {
|
||||
const { clientId, setClientId } = useJukeboxStore();
|
||||
const [inputId, setInputId] = useState(clientId);
|
||||
const [callbackBaseUrl, setCallbackBaseUrl] = useState(() => loadCallbackBaseUrl());
|
||||
const [isRedirecting, setIsRedirecting] = useState(false);
|
||||
const redirectUri = buildRedirectUri(callbackBaseUrl);
|
||||
const localhostOrigin =
|
||||
typeof window !== "undefined" ? `${window.location.protocol}//${window.location.host}` : "";
|
||||
const callbackLooksValid = /^https:\/\/.+/i.test(callbackBaseUrl.trim());
|
||||
|
||||
const handleConnect = async () => {
|
||||
if (!inputId.trim() || !redirectUri) return;
|
||||
saveCallbackBaseUrl(callbackBaseUrl);
|
||||
setClientId(inputId.trim());
|
||||
setIsRedirecting(true);
|
||||
const popup = window.open(
|
||||
"",
|
||||
"soundclaw-spotify-auth",
|
||||
"popup=yes,width=520,height=760,resizable=yes,scrollbars=yes",
|
||||
);
|
||||
if (!popup) {
|
||||
setIsRedirecting(false);
|
||||
return;
|
||||
}
|
||||
popup.document.write("<p style=\"font-family: sans-serif; padding: 24px;\">Redirecting to Spotify...</p>");
|
||||
await startSpotifyAuth(inputId.trim(), redirectUri, popup);
|
||||
setIsRedirecting(false);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="space-y-6 p-6">
|
||||
<div className="rounded-2xl border border-cyan-500/20 bg-cyan-500/5 px-4 py-3 text-sm text-cyan-100">
|
||||
Keep Claw3D open on <code className="rounded bg-slate-900/70 px-1">{localhostOrigin}</code>.
|
||||
Spotify will redirect to your ngrok callback, which sends the auth code back into this window.
|
||||
</div>
|
||||
|
||||
{!callbackLooksValid && callbackBaseUrl.trim().length > 0 && (
|
||||
<div className="rounded-2xl border border-red-500/30 bg-red-500/10 px-4 py-3 text-sm text-red-300">
|
||||
Enter a valid HTTPS ngrok URL, for example <code className="rounded bg-slate-900/70 px-1">https://your-id.ngrok-free.app</code>.
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* What you need card. */}
|
||||
<div className="rounded-2xl border border-amber-500/20 bg-amber-500/5 p-5">
|
||||
<h3 className="mb-3 flex items-center gap-2 text-sm font-semibold text-amber-300">
|
||||
<span>⚠️</span> What you need before connecting
|
||||
</h3>
|
||||
<ol className="space-y-3 text-sm text-slate-300">
|
||||
<li className="flex gap-2">
|
||||
<span className="mt-0.5 flex h-5 w-5 shrink-0 items-center justify-center rounded-full bg-amber-500/20 text-xs font-bold text-amber-300">1</span>
|
||||
<span>
|
||||
Go to{" "}
|
||||
<a
|
||||
href="https://developer.spotify.com/dashboard"
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
className="text-cyan-400 underline underline-offset-2 hover:text-cyan-300"
|
||||
>
|
||||
developer.spotify.com/dashboard
|
||||
</a>{" "}
|
||||
and create an app (or use an existing one).
|
||||
</span>
|
||||
</li>
|
||||
<li className="flex gap-2">
|
||||
<span className="mt-0.5 flex h-5 w-5 shrink-0 items-center justify-center rounded-full bg-amber-500/20 text-xs font-bold text-amber-300">2</span>
|
||||
<span>
|
||||
In your Spotify app settings, add this <strong className="text-white">Redirect URI</strong>:
|
||||
</span>
|
||||
</li>
|
||||
{redirectUri && (
|
||||
<li className="ml-7">
|
||||
<code className="block w-full rounded-lg border border-cyan-500/20 bg-slate-900 px-3 py-2 font-mono text-xs text-cyan-300 break-all">
|
||||
{redirectUri}
|
||||
</code>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => navigator.clipboard.writeText(redirectUri)}
|
||||
className="mt-1.5 text-xs text-slate-500 hover:text-slate-300"
|
||||
>
|
||||
Copy to clipboard
|
||||
</button>
|
||||
</li>
|
||||
)}
|
||||
<li className="flex gap-2">
|
||||
<span className="mt-0.5 flex h-5 w-5 shrink-0 items-center justify-center rounded-full bg-amber-500/20 text-xs font-bold text-amber-300">3</span>
|
||||
<span>Paste your public <strong className="text-white">ngrok URL</strong> below, then use the exact redirect shown here in Spotify.</span>
|
||||
</li>
|
||||
<li className="flex gap-2">
|
||||
<span className="mt-0.5 flex h-5 w-5 shrink-0 items-center justify-center rounded-full bg-amber-500/20 text-xs font-bold text-amber-300">4</span>
|
||||
<span>Keep this local office tab open while authenticating. The popup callback will hand the code back to this page.</span>
|
||||
</li>
|
||||
<li className="flex gap-2">
|
||||
<span className="mt-0.5 flex h-5 w-5 shrink-0 items-center justify-center rounded-full bg-amber-500/20 text-xs font-bold text-amber-300">5</span>
|
||||
<span>Make sure Spotify is open and playing on at least one device before using playback controls.</span>
|
||||
</li>
|
||||
</ol>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<label className="block text-sm font-medium text-slate-300">
|
||||
ngrok Public URL
|
||||
</label>
|
||||
<input
|
||||
type="url"
|
||||
value={callbackBaseUrl}
|
||||
onChange={(e) => setCallbackBaseUrl(e.target.value)}
|
||||
placeholder="https://your-id.ngrok-free.app"
|
||||
className="w-full rounded-xl border border-white/10 bg-slate-900 px-4 py-2.5 font-mono text-sm text-white placeholder-slate-600 focus:border-cyan-500/50 focus:outline-none focus:ring-1 focus:ring-cyan-500/30"
|
||||
/>
|
||||
<p className="text-xs text-slate-500">
|
||||
This is only used for the Spotify OAuth callback bridge. Your app can stay on {localhostOrigin}.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Client ID input. */}
|
||||
<div className="space-y-2">
|
||||
<label className="block text-sm font-medium text-slate-300">
|
||||
Spotify Client ID
|
||||
</label>
|
||||
<input
|
||||
type="text"
|
||||
value={inputId}
|
||||
onChange={(e) => setInputId(e.target.value)}
|
||||
placeholder="e.g. 1a2b3c4d5e6f…"
|
||||
className="w-full rounded-xl border border-white/10 bg-slate-900 px-4 py-2.5 font-mono text-sm text-white placeholder-slate-600 focus:border-cyan-500/50 focus:outline-none focus:ring-1 focus:ring-cyan-500/30"
|
||||
/>
|
||||
<p className="text-xs text-slate-500">
|
||||
Stored locally in your browser. Never sent to any server other than Spotify.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<button
|
||||
type="button"
|
||||
disabled={!inputId.trim() || !redirectUri || !callbackLooksValid || isRedirecting}
|
||||
onClick={handleConnect}
|
||||
className="w-full rounded-xl bg-[#1DB954] py-3 text-sm font-semibold text-black transition hover:bg-[#1ed760] active:scale-[.98] disabled:cursor-not-allowed disabled:opacity-40"
|
||||
>
|
||||
{isRedirecting ? "Opening Spotify…" : "Connect with Spotify"}
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Player view — shown after authentication
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function PlayerView() {
|
||||
const {
|
||||
playerState,
|
||||
searchResults,
|
||||
searchQuery,
|
||||
isSearching,
|
||||
isLoadingPlayer,
|
||||
error,
|
||||
refreshPlayer,
|
||||
search,
|
||||
setSearchQuery,
|
||||
play,
|
||||
pause,
|
||||
resume,
|
||||
next,
|
||||
previous,
|
||||
volume,
|
||||
disconnect,
|
||||
} = useJukeboxStore();
|
||||
|
||||
const searchDebounce = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
|
||||
// Poll player state every 5 seconds.
|
||||
useEffect(() => {
|
||||
refreshPlayer();
|
||||
const id = window.setInterval(() => { void refreshPlayer(); }, 5000);
|
||||
return () => window.clearInterval(id);
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
|
||||
const handleSearchChange = (value: string) => {
|
||||
setSearchQuery(value);
|
||||
if (searchDebounce.current) clearTimeout(searchDebounce.current);
|
||||
searchDebounce.current = setTimeout(() => {
|
||||
if (value.trim()) void search(value);
|
||||
}, 400);
|
||||
};
|
||||
|
||||
const track = playerState?.track;
|
||||
const albumArt = track?.album.images[0]?.url ?? null;
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-4 p-6">
|
||||
{error && (
|
||||
<div className="rounded-xl border border-red-500/20 bg-red-500/5 px-4 py-3 text-sm text-red-400">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Now playing. */}
|
||||
<div className="rounded-2xl border border-white/5 bg-slate-900/60 p-4">
|
||||
<div className="mb-3 font-mono text-[10px] uppercase tracking-[0.2em] text-slate-500">
|
||||
Now playing
|
||||
</div>
|
||||
{isLoadingPlayer && !track ? (
|
||||
<div className="flex items-center gap-3 text-slate-500">
|
||||
<div className="h-4 w-4 animate-spin rounded-full border-2 border-slate-600 border-t-cyan-500" />
|
||||
<span className="text-sm">Loading player…</span>
|
||||
</div>
|
||||
) : track ? (
|
||||
<div className="flex items-center gap-4">
|
||||
{albumArt && (
|
||||
// eslint-disable-next-line @next/next/no-img-element
|
||||
<img
|
||||
src={albumArt}
|
||||
alt={track.album.name}
|
||||
className="h-14 w-14 shrink-0 rounded-lg object-cover shadow-lg"
|
||||
/>
|
||||
)}
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="truncate font-semibold text-white">{track.name}</div>
|
||||
<div className="truncate text-sm text-slate-400">
|
||||
{track.artists.map((a) => a.name).join(", ")}
|
||||
</div>
|
||||
<div className="truncate text-xs text-slate-600">{track.album.name}</div>
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<p className="text-sm text-slate-500">
|
||||
No active playback. Open Spotify on a device first, then hit play.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{/* Transport controls. */}
|
||||
<div className="mt-4 flex items-center justify-center gap-4">
|
||||
<ControlButton icon="⏮" onClick={() => void previous()} title="Previous" />
|
||||
{playerState?.isPlaying ? (
|
||||
<ControlButton icon="⏸" onClick={() => void pause()} title="Pause" large />
|
||||
) : (
|
||||
<ControlButton icon="▶" onClick={() => void resume()} title="Play" large />
|
||||
)}
|
||||
<ControlButton icon="⏭" onClick={() => void next()} title="Next" />
|
||||
</div>
|
||||
|
||||
{/* Volume. */}
|
||||
{playerState && (
|
||||
<div className="mt-4 flex items-center gap-3">
|
||||
<span className="text-sm text-slate-500">🔈</span>
|
||||
<input
|
||||
type="range"
|
||||
min={0}
|
||||
max={100}
|
||||
value={playerState.volumePercent}
|
||||
onChange={(e) => void volume(Number(e.target.value))}
|
||||
className="h-1.5 w-full cursor-pointer accent-cyan-400"
|
||||
/>
|
||||
<span className="w-8 text-right font-mono text-xs text-slate-500">
|
||||
{playerState.volumePercent}%
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Search. */}
|
||||
<div>
|
||||
<div className="mb-2 font-mono text-[10px] uppercase tracking-[0.2em] text-slate-500">
|
||||
Search tracks
|
||||
</div>
|
||||
<div className="relative">
|
||||
<input
|
||||
type="text"
|
||||
value={searchQuery}
|
||||
onChange={(e) => handleSearchChange(e.target.value)}
|
||||
placeholder="Artist, song, or album…"
|
||||
className="w-full rounded-xl border border-white/10 bg-slate-900 py-2.5 pl-4 pr-10 text-sm text-white placeholder-slate-600 focus:border-cyan-500/50 focus:outline-none focus:ring-1 focus:ring-cyan-500/30"
|
||||
/>
|
||||
{isSearching && (
|
||||
<div className="absolute right-3 top-1/2 -translate-y-1/2">
|
||||
<div className="h-3.5 w-3.5 animate-spin rounded-full border-2 border-slate-600 border-t-cyan-500" />
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{searchResults.length > 0 && (
|
||||
<ul className="mt-2 divide-y divide-white/5 overflow-hidden rounded-xl border border-white/5 bg-slate-900/60">
|
||||
{searchResults.map((track) => (
|
||||
<SearchResult key={track.id} track={track} onPlay={() => void play(track.uri)} />
|
||||
))}
|
||||
</ul>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Disconnect. */}
|
||||
<div className="pt-2 text-center">
|
||||
<button
|
||||
type="button"
|
||||
onClick={disconnect}
|
||||
className="text-xs text-slate-600 underline underline-offset-2 hover:text-slate-400"
|
||||
>
|
||||
Disconnect Spotify
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Sub-components
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function ControlButton({
|
||||
icon,
|
||||
onClick,
|
||||
title,
|
||||
large,
|
||||
}: {
|
||||
icon: string;
|
||||
onClick: () => void;
|
||||
title: string;
|
||||
large?: boolean;
|
||||
}) {
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
title={title}
|
||||
onClick={onClick}
|
||||
className={`flex items-center justify-center rounded-full border border-white/10 text-white transition hover:bg-white/10 active:scale-95 ${
|
||||
large ? "h-11 w-11 text-lg" : "h-9 w-9 text-sm"
|
||||
}`}
|
||||
>
|
||||
{icon}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
function SearchResult({
|
||||
track,
|
||||
onPlay,
|
||||
}: {
|
||||
track: SpotifyTrack;
|
||||
onPlay: () => void;
|
||||
}) {
|
||||
const art = track.album.images[track.album.images.length - 1]?.url ?? null;
|
||||
return (
|
||||
<li className="flex items-center gap-3 px-4 py-3 transition hover:bg-white/5">
|
||||
{art && (
|
||||
// eslint-disable-next-line @next/next/no-img-element
|
||||
<img src={art} alt={track.album.name} className="h-9 w-9 shrink-0 rounded object-cover" />
|
||||
)}
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="truncate text-sm font-medium text-white">{track.name}</div>
|
||||
<div className="truncate text-xs text-slate-400">
|
||||
{track.artists.map((a) => a.name).join(", ")} · {track.album.name}
|
||||
</div>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onPlay}
|
||||
className="shrink-0 rounded-full border border-cyan-500/30 px-3 py-1 text-xs font-medium text-cyan-400 transition hover:bg-cyan-500/10"
|
||||
>
|
||||
Play
|
||||
</button>
|
||||
</li>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
"use client";
|
||||
|
||||
// Thin Spotify Web API wrapper used by the jukebox panel.
|
||||
|
||||
export type SpotifyTrack = {
|
||||
id: string;
|
||||
name: string;
|
||||
uri: string;
|
||||
durationMs: number;
|
||||
artists: { name: string }[];
|
||||
album: { name: string; images: { url: string; width: number; height: number }[] };
|
||||
};
|
||||
|
||||
export type PlayerState = {
|
||||
isPlaying: boolean;
|
||||
progressMs: number;
|
||||
track: SpotifyTrack | null;
|
||||
volumePercent: number;
|
||||
deviceId: string | null;
|
||||
};
|
||||
|
||||
type RawTrack = {
|
||||
id: string;
|
||||
name: string;
|
||||
uri: string;
|
||||
duration_ms: number;
|
||||
artists: { name: string }[];
|
||||
album: { name: string; images: { url: string; width: number; height: number }[] };
|
||||
};
|
||||
|
||||
type RawPlayerState = {
|
||||
is_playing: boolean;
|
||||
progress_ms: number;
|
||||
device: { id: string; volume_percent: number };
|
||||
item: RawTrack | null;
|
||||
};
|
||||
|
||||
const BASE = "https://api.spotify.com/v1";
|
||||
|
||||
const headers = (token: string) => ({
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Content-Type": "application/json",
|
||||
});
|
||||
|
||||
const mapTrack = (raw: RawTrack): SpotifyTrack => ({
|
||||
id: raw.id,
|
||||
name: raw.name,
|
||||
uri: raw.uri,
|
||||
durationMs: raw.duration_ms,
|
||||
artists: raw.artists,
|
||||
album: raw.album,
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Player state
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const fetchPlayerState = async (token: string): Promise<PlayerState | null> => {
|
||||
const res = await fetch(`${BASE}/me/player`, { headers: headers(token) });
|
||||
if (res.status === 204 || !res.ok) return null;
|
||||
const data = (await res.json()) as RawPlayerState;
|
||||
return {
|
||||
isPlaying: data.is_playing,
|
||||
progressMs: data.progress_ms,
|
||||
track: data.item ? mapTrack(data.item) : null,
|
||||
volumePercent: data.device?.volume_percent ?? 50,
|
||||
deviceId: data.device?.id ?? null,
|
||||
};
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Search
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const searchTracks = async (token: string, query: string): Promise<SpotifyTrack[]> => {
|
||||
const params = new URLSearchParams({ q: query, type: "track", limit: "10" });
|
||||
const res = await fetch(`${BASE}/search?${params}`, { headers: headers(token) });
|
||||
if (!res.ok) return [];
|
||||
const data = await res.json() as { tracks: { items: RawTrack[] } };
|
||||
return (data.tracks?.items ?? []).map(mapTrack);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Playback controls
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const playTrack = async (token: string, uri: string, deviceId?: string | null): Promise<void> => {
|
||||
const params = deviceId ? `?device_id=${deviceId}` : "";
|
||||
await fetch(`${BASE}/me/player/play${params}`, {
|
||||
method: "PUT",
|
||||
headers: headers(token),
|
||||
body: JSON.stringify({ uris: [uri] }),
|
||||
});
|
||||
};
|
||||
|
||||
export const pausePlayback = async (token: string): Promise<void> => {
|
||||
await fetch(`${BASE}/me/player/pause`, { method: "PUT", headers: headers(token) });
|
||||
};
|
||||
|
||||
export const resumePlayback = async (token: string): Promise<void> => {
|
||||
await fetch(`${BASE}/me/player/play`, { method: "PUT", headers: headers(token) });
|
||||
};
|
||||
|
||||
export const skipToNext = async (token: string): Promise<void> => {
|
||||
await fetch(`${BASE}/me/player/next`, { method: "POST", headers: headers(token) });
|
||||
};
|
||||
|
||||
export const skipToPrevious = async (token: string): Promise<void> => {
|
||||
await fetch(`${BASE}/me/player/previous`, { method: "POST", headers: headers(token) });
|
||||
};
|
||||
|
||||
export const setVolume = async (token: string, volumePercent: number): Promise<void> => {
|
||||
const params = new URLSearchParams({ volume_percent: String(Math.round(volumePercent)) });
|
||||
await fetch(`${BASE}/me/player/volume?${params}`, { method: "PUT", headers: headers(token) });
|
||||
};
|
||||
@@ -0,0 +1,182 @@
|
||||
"use client";
|
||||
|
||||
import { create } from "zustand";
|
||||
import { loadToken, clearToken, loadClientId, saveClientId } from "./auth";
|
||||
import {
|
||||
fetchPlayerState,
|
||||
searchTracks,
|
||||
playTrack,
|
||||
pausePlayback,
|
||||
resumePlayback,
|
||||
skipToNext,
|
||||
skipToPrevious,
|
||||
setVolume,
|
||||
type PlayerState,
|
||||
type SpotifyTrack,
|
||||
} from "./spotifyApi";
|
||||
|
||||
const SOUNDCLAW_PLAYBACK_STARTED_EVENT = "soundclaw:playback-started";
|
||||
|
||||
const emitPlaybackStarted = () => {
|
||||
if (typeof window === "undefined") return;
|
||||
window.dispatchEvent(
|
||||
new CustomEvent(SOUNDCLAW_PLAYBACK_STARTED_EVENT, {
|
||||
detail: { startedAt: Date.now() },
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
type JukeboxView = "setup" | "player";
|
||||
|
||||
type JukeboxStore = {
|
||||
// Auth state.
|
||||
token: string | null;
|
||||
clientId: string;
|
||||
view: JukeboxView;
|
||||
|
||||
// Player state.
|
||||
playerState: PlayerState | null;
|
||||
searchResults: SpotifyTrack[];
|
||||
searchQuery: string;
|
||||
isSearching: boolean;
|
||||
isLoadingPlayer: boolean;
|
||||
error: string | null;
|
||||
|
||||
// Actions.
|
||||
init: () => void;
|
||||
setClientId: (id: string) => void;
|
||||
setToken: (token: string) => void;
|
||||
disconnect: () => void;
|
||||
refreshPlayer: () => Promise<void>;
|
||||
search: (query: string) => Promise<void>;
|
||||
setSearchQuery: (q: string) => void;
|
||||
play: (uri: string) => Promise<void>;
|
||||
pause: () => Promise<void>;
|
||||
resume: () => Promise<void>;
|
||||
next: () => Promise<void>;
|
||||
previous: () => Promise<void>;
|
||||
volume: (percent: number) => Promise<void>;
|
||||
};
|
||||
|
||||
export const useJukeboxStore = create<JukeboxStore>((set, get) => ({
|
||||
token: null,
|
||||
clientId: "",
|
||||
view: "setup",
|
||||
playerState: null,
|
||||
searchResults: [],
|
||||
searchQuery: "",
|
||||
isSearching: false,
|
||||
isLoadingPlayer: false,
|
||||
error: null,
|
||||
|
||||
init: () => {
|
||||
const token = loadToken();
|
||||
const clientId = loadClientId();
|
||||
set({
|
||||
token,
|
||||
clientId,
|
||||
view: token ? "player" : "setup",
|
||||
});
|
||||
},
|
||||
|
||||
setClientId: (id) => {
|
||||
saveClientId(id);
|
||||
set({ clientId: id });
|
||||
},
|
||||
|
||||
setToken: (token) => {
|
||||
set({ token, view: "player" });
|
||||
},
|
||||
|
||||
disconnect: () => {
|
||||
clearToken();
|
||||
set({ token: null, view: "setup", playerState: null, searchResults: [], searchQuery: "" });
|
||||
},
|
||||
|
||||
refreshPlayer: async () => {
|
||||
const { token } = get();
|
||||
if (!token) return;
|
||||
set({ isLoadingPlayer: true, error: null });
|
||||
try {
|
||||
const playerState = await fetchPlayerState(token);
|
||||
set({ playerState, isLoadingPlayer: false });
|
||||
} catch {
|
||||
set({ isLoadingPlayer: false, error: "Could not reach Spotify." });
|
||||
}
|
||||
},
|
||||
|
||||
search: async (query) => {
|
||||
const { token } = get();
|
||||
if (!token || !query.trim()) return;
|
||||
set({ isSearching: true, error: null });
|
||||
try {
|
||||
const results = await searchTracks(token, query);
|
||||
set({ searchResults: results, isSearching: false });
|
||||
} catch {
|
||||
set({ isSearching: false, error: "Search failed." });
|
||||
}
|
||||
},
|
||||
|
||||
setSearchQuery: (q) => set({ searchQuery: q }),
|
||||
|
||||
play: async (uri) => {
|
||||
const { token, playerState } = get();
|
||||
if (!token) return;
|
||||
set({ error: null });
|
||||
try {
|
||||
await playTrack(token, uri, playerState?.deviceId);
|
||||
// Brief delay for Spotify to update state.
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
await get().refreshPlayer();
|
||||
emitPlaybackStarted();
|
||||
} catch {
|
||||
set({ error: "Playback failed. Make sure Spotify is open on a device." });
|
||||
}
|
||||
},
|
||||
|
||||
pause: async () => {
|
||||
const { token } = get();
|
||||
if (!token) return;
|
||||
await pausePlayback(token);
|
||||
set((s) => ({
|
||||
playerState: s.playerState ? { ...s.playerState, isPlaying: false } : null,
|
||||
}));
|
||||
},
|
||||
|
||||
resume: async () => {
|
||||
const { token } = get();
|
||||
if (!token) return;
|
||||
await resumePlayback(token);
|
||||
set((s) => ({
|
||||
playerState: s.playerState ? { ...s.playerState, isPlaying: true } : null,
|
||||
}));
|
||||
emitPlaybackStarted();
|
||||
},
|
||||
|
||||
next: async () => {
|
||||
const { token } = get();
|
||||
if (!token) return;
|
||||
await skipToNext(token);
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
await get().refreshPlayer();
|
||||
},
|
||||
|
||||
previous: async () => {
|
||||
const { token } = get();
|
||||
if (!token) return;
|
||||
await skipToPrevious(token);
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
await get().refreshPlayer();
|
||||
},
|
||||
|
||||
volume: async (percent) => {
|
||||
const { token } = get();
|
||||
if (!token) return;
|
||||
await setVolume(token, percent);
|
||||
set((s) => ({
|
||||
playerState: s.playerState ? { ...s.playerState, volumePercent: percent } : null,
|
||||
}));
|
||||
},
|
||||
}));
|
||||
|
||||
export const SOUNDCLAW_PLAYBACK_STARTED_EVENT_NAME = SOUNDCLAW_PLAYBACK_STARTED_EVENT;
|
||||
+109
-51
@@ -40,10 +40,7 @@ import {
|
||||
resolveOfficeStandupDirective,
|
||||
resolveOfficeTextDirective,
|
||||
} from "@/lib/office/deskDirectives";
|
||||
import {
|
||||
extractText,
|
||||
extractThinking,
|
||||
} from "@/lib/text/message-extract";
|
||||
import { extractText, extractThinking } from "@/lib/text/message-extract";
|
||||
import { randomUUID } from "@/lib/uuid";
|
||||
|
||||
// Office animation is derived in two passes:
|
||||
@@ -120,9 +117,11 @@ export type OfficeAnimationTriggerState = {
|
||||
export type OfficeAnimationState = {
|
||||
awaitingApprovalByAgentId: BooleanByAgentId;
|
||||
cleaningCues: OfficeCleaningCue[];
|
||||
danceUntilByAgentId: NumberByAgentId;
|
||||
deskHoldByAgentId: BooleanByAgentId;
|
||||
githubHoldByAgentId: BooleanByAgentId;
|
||||
gymHoldByAgentId: BooleanByAgentId;
|
||||
jukeboxHoldByAgentId: BooleanByAgentId;
|
||||
manualGymUntilByAgentId: NumberByAgentId;
|
||||
pendingStandupRequest: OfficeStandupTriggerRequest | null;
|
||||
phoneBoothHoldByAgentId: BooleanByAgentId;
|
||||
@@ -136,14 +135,11 @@ export type OfficeAnimationState = {
|
||||
workingUntilByAgentId: NumberByAgentId;
|
||||
};
|
||||
|
||||
const emptyObject = <T extends Record<string, unknown>>(): T => ({} as T);
|
||||
const emptyObject = <T extends Record<string, unknown>>(): T => ({}) as T;
|
||||
|
||||
const normalizeCommandText = (value: string | null | undefined): string => {
|
||||
if (!value) return "";
|
||||
return value
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(/\s+/g, " ");
|
||||
return value.trim().toLowerCase().replace(/\s+/g, " ");
|
||||
};
|
||||
|
||||
const buildStableLatestRequestSeed = (value: string): string => {
|
||||
@@ -167,7 +163,8 @@ const pruneStringMap = (
|
||||
): StringByAgentId =>
|
||||
Object.fromEntries(
|
||||
Object.entries(source).filter(
|
||||
([agentId, value]) => activeAgentIds.has(agentId) && value.trim().length > 0,
|
||||
([agentId, value]) =>
|
||||
activeAgentIds.has(agentId) && value.trim().length > 0,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -181,7 +178,8 @@ const prunePhoneCallMap = (
|
||||
activeAgentIds.has(agentId) &&
|
||||
Boolean(request?.callee?.trim()) &&
|
||||
(request.phase === "needs_message" ||
|
||||
(request.phase === "ready_to_call" && Boolean(request.message?.trim()))),
|
||||
(request.phase === "ready_to_call" &&
|
||||
Boolean(request.message?.trim()))),
|
||||
),
|
||||
);
|
||||
|
||||
@@ -195,7 +193,8 @@ const pruneTextMessageMap = (
|
||||
activeAgentIds.has(agentId) &&
|
||||
Boolean(request?.recipient?.trim()) &&
|
||||
(request.phase === "needs_message" ||
|
||||
(request.phase === "ready_to_send" && Boolean(request.message?.trim()))),
|
||||
(request.phase === "ready_to_send" &&
|
||||
Boolean(request.message?.trim()))),
|
||||
),
|
||||
);
|
||||
|
||||
@@ -220,7 +219,9 @@ const resolveMessageRole = (message: unknown): string | null => {
|
||||
return typeof role === "string" ? role : null;
|
||||
};
|
||||
|
||||
const resolveChatPayloadRole = (payload: ChatEventPayload | undefined): string | null => {
|
||||
const resolveChatPayloadRole = (
|
||||
payload: ChatEventPayload | undefined,
|
||||
): string | null => {
|
||||
if (!payload) return null;
|
||||
const messageRole = resolveMessageRole(payload.message);
|
||||
if (messageRole) return messageRole;
|
||||
@@ -231,19 +232,20 @@ const resolveChatPayloadRole = (payload: ChatEventPayload | undefined): string |
|
||||
return typeof payloadRole === "string" ? payloadRole : null;
|
||||
};
|
||||
|
||||
const isUserLikeChatRole = (role: string | null, state: ChatEventPayload["state"]): boolean => {
|
||||
const isUserLikeChatRole = (
|
||||
role: string | null,
|
||||
state: ChatEventPayload["state"],
|
||||
): boolean => {
|
||||
if (role === "user" || role === "human" || role === "input") return true;
|
||||
if (role === "system") return state === "final";
|
||||
return role === null && state === "final";
|
||||
};
|
||||
|
||||
const resolveLatestDirective = <TDirective>(
|
||||
params: {
|
||||
const resolveLatestDirective = <TDirective>(params: {
|
||||
lastUserMessage: string | null | undefined;
|
||||
transcriptEntries: TranscriptEntry[] | undefined;
|
||||
resolver: (value: string | null | undefined) => TDirective | null;
|
||||
},
|
||||
): LatestDirective<TDirective> | null => {
|
||||
}): LatestDirective<TDirective> | null => {
|
||||
const latestMessageDirective = params.resolver(params.lastUserMessage);
|
||||
if (latestMessageDirective) {
|
||||
const text = params.lastUserMessage?.trim() ?? "";
|
||||
@@ -253,10 +255,17 @@ const resolveLatestDirective = <TDirective>(
|
||||
text,
|
||||
};
|
||||
}
|
||||
if (!Array.isArray(params.transcriptEntries) || params.transcriptEntries.length === 0) {
|
||||
if (
|
||||
!Array.isArray(params.transcriptEntries) ||
|
||||
params.transcriptEntries.length === 0
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
for (let index = params.transcriptEntries.length - 1; index >= 0; index -= 1) {
|
||||
for (
|
||||
let index = params.transcriptEntries.length - 1;
|
||||
index >= 0;
|
||||
index -= 1
|
||||
) {
|
||||
const entry = params.transcriptEntries[index];
|
||||
if (!entry || entry.role !== "user") continue;
|
||||
const directive = params.resolver(entry.text);
|
||||
@@ -270,17 +279,23 @@ const resolveLatestDirective = <TDirective>(
|
||||
return null;
|
||||
};
|
||||
|
||||
const isTransientBoothRequestFresh = (requestedAt: number, nowMs: number): boolean =>
|
||||
nowMs - requestedAt <= TRANSIENT_BOOTH_RESTORE_MAX_AGE_MS;
|
||||
const isTransientBoothRequestFresh = (
|
||||
requestedAt: number,
|
||||
nowMs: number,
|
||||
): boolean => nowMs - requestedAt <= TRANSIENT_BOOTH_RESTORE_MAX_AGE_MS;
|
||||
|
||||
const maybeResolveCompletedPhoneCallRequest = (
|
||||
current: OfficePhoneCallRequest | null,
|
||||
line: string,
|
||||
): OfficePhoneCallRequest | null => {
|
||||
if (!current) return null;
|
||||
const match = line.match(/^\[phone booth\]\s*Call with\s+(.+)\s+finished\.$/i);
|
||||
const match = line.match(
|
||||
/^\[phone booth\]\s*Call with\s+(.+)\s+finished\.$/i,
|
||||
);
|
||||
if (!match) return current;
|
||||
return normalizeCommandText(match[1]) === normalizeCommandText(current.callee) ? null : current;
|
||||
return normalizeCommandText(match[1]) === normalizeCommandText(current.callee)
|
||||
? null
|
||||
: current;
|
||||
};
|
||||
|
||||
const maybeResolveCompletedTextMessageRequest = (
|
||||
@@ -288,9 +303,12 @@ const maybeResolveCompletedTextMessageRequest = (
|
||||
line: string,
|
||||
): OfficeTextMessageRequest | null => {
|
||||
if (!current) return null;
|
||||
const match = line.match(/^\[messaging booth\]\s*Message to\s+(.+)\s+sent\.$/i);
|
||||
const match = line.match(
|
||||
/^\[messaging booth\]\s*Message to\s+(.+)\s+sent\.$/i,
|
||||
);
|
||||
if (!match) return current;
|
||||
return normalizeCommandText(match[1]) === normalizeCommandText(current.recipient)
|
||||
return normalizeCommandText(match[1]) ===
|
||||
normalizeCommandText(current.recipient)
|
||||
? null
|
||||
: current;
|
||||
};
|
||||
@@ -361,7 +379,9 @@ const resolveLatestPhoneCallRequest = (params: {
|
||||
}
|
||||
}
|
||||
if (!current) return null;
|
||||
return isTransientBoothRequestFresh(current.requestedAt, params.nowMs) ? current : null;
|
||||
return isTransientBoothRequestFresh(current.requestedAt, params.nowMs)
|
||||
? current
|
||||
: null;
|
||||
};
|
||||
|
||||
const resolveLatestTextMessageRequest = (params: {
|
||||
@@ -430,7 +450,9 @@ const resolveLatestTextMessageRequest = (params: {
|
||||
}
|
||||
}
|
||||
if (!current) return null;
|
||||
return isTransientBoothRequestFresh(current.requestedAt, params.nowMs) ? current : null;
|
||||
return isTransientBoothRequestFresh(current.requestedAt, params.nowMs)
|
||||
? current
|
||||
: null;
|
||||
};
|
||||
|
||||
const resolveAgentIdForSessionKey = (
|
||||
@@ -439,7 +461,9 @@ const resolveAgentIdForSessionKey = (
|
||||
): string | null => {
|
||||
const trimmed = sessionKey?.trim() ?? "";
|
||||
if (!trimmed) return null;
|
||||
const matched = agents.find((agent) => isSameSessionKey(agent.sessionKey, trimmed));
|
||||
const matched = agents.find((agent) =>
|
||||
isSameSessionKey(agent.sessionKey, trimmed),
|
||||
);
|
||||
if (matched) return matched.agentId;
|
||||
return parseAgentIdFromSessionKey(trimmed);
|
||||
};
|
||||
@@ -522,11 +546,14 @@ const resolvePhoneCallFollowUpRequest = (params: {
|
||||
const message = params.message.trim();
|
||||
if (!message) return null;
|
||||
return {
|
||||
key: buildPhoneCallDirectiveKey({
|
||||
key: buildPhoneCallDirectiveKey(
|
||||
{
|
||||
callee: params.current.callee,
|
||||
phase: "ready_to_call",
|
||||
message,
|
||||
}, params.requestSeed ?? String(params.requestedAt)),
|
||||
},
|
||||
params.requestSeed ?? String(params.requestedAt),
|
||||
),
|
||||
callee: params.current.callee,
|
||||
message,
|
||||
phase: "ready_to_call",
|
||||
@@ -587,7 +614,10 @@ const pruneOfficeAnimationTriggerState = (
|
||||
state.githubDirectiveKeyByAgentId,
|
||||
activeAgentIds,
|
||||
),
|
||||
githubHoldByAgentId: pruneBooleanMap(state.githubHoldByAgentId, activeAgentIds),
|
||||
githubHoldByAgentId: pruneBooleanMap(
|
||||
state.githubHoldByAgentId,
|
||||
activeAgentIds,
|
||||
),
|
||||
gymCooldownUntilByAgentId: pruneFutureMap(
|
||||
state.gymCooldownUntilByAgentId,
|
||||
activeAgentIds,
|
||||
@@ -606,7 +636,10 @@ const pruneOfficeAnimationTriggerState = (
|
||||
state.qaDirectiveKeyByAgentId,
|
||||
activeAgentIds,
|
||||
),
|
||||
phoneCallByAgentId: prunePhoneCallMap(state.phoneCallByAgentId, activeAgentIds),
|
||||
phoneCallByAgentId: prunePhoneCallMap(
|
||||
state.phoneCallByAgentId,
|
||||
activeAgentIds,
|
||||
),
|
||||
phoneCallDirectiveKeyByAgentId: pruneStringMap(
|
||||
state.phoneCallDirectiveKeyByAgentId,
|
||||
activeAgentIds,
|
||||
@@ -686,7 +719,10 @@ const recordThinkingActivity = (
|
||||
nowMs: number,
|
||||
): NumberByAgentId => ({
|
||||
...current,
|
||||
[agentId]: Math.max(current[agentId] ?? 0, nowMs + THINKING_ACTIVITY_LATCH_MS),
|
||||
[agentId]: Math.max(
|
||||
current[agentId] ?? 0,
|
||||
nowMs + THINKING_ACTIVITY_LATCH_MS,
|
||||
),
|
||||
});
|
||||
|
||||
const applyUserMessageTriggers = (params: {
|
||||
@@ -717,7 +753,8 @@ const applyUserMessageTriggers = (params: {
|
||||
if (githubDirective) {
|
||||
const directiveKey = normalizeCommandText(params.message);
|
||||
const isSuppressed =
|
||||
next.suppressedGithubDirectiveKeyByAgentId[params.agentId] === directiveKey;
|
||||
next.suppressedGithubDirectiveKeyByAgentId[params.agentId] ===
|
||||
directiveKey;
|
||||
next = {
|
||||
...next,
|
||||
githubDirectiveKeyByAgentId: {
|
||||
@@ -769,10 +806,7 @@ const applyUserMessageTriggers = (params: {
|
||||
},
|
||||
};
|
||||
}
|
||||
if (
|
||||
params.agentId === "main" &&
|
||||
intentSnapshot.standup === "standup"
|
||||
) {
|
||||
if (params.agentId === "main" && intentSnapshot.standup === "standup") {
|
||||
const requestKey = normalizeCommandText(params.message);
|
||||
if (next.pendingStandupRequest?.key !== requestKey) {
|
||||
next = {
|
||||
@@ -862,7 +896,8 @@ const applyUserMessageTriggers = (params: {
|
||||
return next;
|
||||
};
|
||||
|
||||
export const createOfficeAnimationTriggerState = (): OfficeAnimationTriggerState => ({
|
||||
export const createOfficeAnimationTriggerState =
|
||||
(): OfficeAnimationTriggerState => ({
|
||||
cleaningCues: [],
|
||||
deskDirectiveKeyByAgentId: emptyObject(),
|
||||
deskHoldByAgentId: emptyObject(),
|
||||
@@ -888,7 +923,7 @@ export const createOfficeAnimationTriggerState = (): OfficeAnimationTriggerState
|
||||
textMessageDirectiveKeyByAgentId: emptyObject(),
|
||||
thinkingUntilByAgentId: emptyObject(),
|
||||
workingUntilByAgentId: emptyObject(),
|
||||
});
|
||||
});
|
||||
|
||||
export const reduceOfficeAnimationTriggerEvent = (params: {
|
||||
agents: AgentState[];
|
||||
@@ -897,7 +932,11 @@ export const reduceOfficeAnimationTriggerEvent = (params: {
|
||||
state: OfficeAnimationTriggerState;
|
||||
}): OfficeAnimationTriggerState => {
|
||||
const nowMs = params.nowMs ?? Date.now();
|
||||
let next = pruneOfficeAnimationTriggerState(params.state, params.agents, nowMs);
|
||||
let next = pruneOfficeAnimationTriggerState(
|
||||
params.state,
|
||||
params.agents,
|
||||
nowMs,
|
||||
);
|
||||
const kind = classifyGatewayEventKind(params.event.event);
|
||||
|
||||
if (kind === "runtime-chat") {
|
||||
@@ -908,7 +947,8 @@ export const reduceOfficeAnimationTriggerEvent = (params: {
|
||||
);
|
||||
if (!payload || !agentId) return next;
|
||||
const messageText = extractText(payload.message)?.trim() ?? "";
|
||||
const thinkingText = extractThinking(payload.message ?? payload)?.trim() ?? "";
|
||||
const thinkingText =
|
||||
extractThinking(payload.message ?? payload)?.trim() ?? "";
|
||||
const role = resolveChatPayloadRole(payload);
|
||||
if (payload.runId) {
|
||||
next = {
|
||||
@@ -1015,7 +1055,9 @@ export const reduceOfficeAnimationTriggerEvent = (params: {
|
||||
|
||||
const resolved = parseExecApprovalResolved(params.event);
|
||||
if (resolved) {
|
||||
const approvalAgentId = params.agents.find((agent) => agent.awaitingUserInput)?.agentId;
|
||||
const approvalAgentId = params.agents.find(
|
||||
(agent) => agent.awaitingUserInput,
|
||||
)?.agentId;
|
||||
if (approvalAgentId) {
|
||||
next = {
|
||||
...next,
|
||||
@@ -1039,7 +1081,11 @@ export const reconcileOfficeAnimationTriggerState = (params: {
|
||||
// Reconciliation is the durable source of truth. It replays the latest user-visible intent
|
||||
// from current agent state so recovered history can restore holds even when chat events were missed.
|
||||
const nowMs = params.nowMs ?? Date.now();
|
||||
const next = pruneOfficeAnimationTriggerState(params.state, params.agents, nowMs);
|
||||
const next = pruneOfficeAnimationTriggerState(
|
||||
params.state,
|
||||
params.agents,
|
||||
nowMs,
|
||||
);
|
||||
|
||||
const activeAgentIds = new Set(params.agents.map((agent) => agent.agentId));
|
||||
const currentImmediateGymKeys = pruneStringMap(
|
||||
@@ -1100,7 +1146,8 @@ export const reconcileOfficeAnimationTriggerState = (params: {
|
||||
});
|
||||
if (githubDirective) {
|
||||
githubDirectiveKeyByAgentId[agentId] = githubDirective.key;
|
||||
const suppressedKey = next.suppressedGithubDirectiveKeyByAgentId[agentId] ?? "";
|
||||
const suppressedKey =
|
||||
next.suppressedGithubDirectiveKeyByAgentId[agentId] ?? "";
|
||||
if (
|
||||
githubDirective.directive !== "release" &&
|
||||
suppressedKey !== githubDirective.key
|
||||
@@ -1118,8 +1165,12 @@ export const reconcileOfficeAnimationTriggerState = (params: {
|
||||
});
|
||||
if (qaDirective) {
|
||||
qaDirectiveKeyByAgentId[agentId] = qaDirective.key;
|
||||
const suppressedKey = next.suppressedQaDirectiveKeyByAgentId[agentId] ?? "";
|
||||
if (qaDirective.directive !== "release" && suppressedKey !== qaDirective.key) {
|
||||
const suppressedKey =
|
||||
next.suppressedQaDirectiveKeyByAgentId[agentId] ?? "";
|
||||
if (
|
||||
qaDirective.directive !== "release" &&
|
||||
suppressedKey !== qaDirective.key
|
||||
) {
|
||||
qaHoldByAgentId[agentId] = true;
|
||||
}
|
||||
} else if (next.qaHoldByAgentId[agentId]) {
|
||||
@@ -1190,7 +1241,9 @@ export const reconcileOfficeAnimationTriggerState = (params: {
|
||||
previous: next.sessionEpochSnapshot,
|
||||
agents: params.agents,
|
||||
});
|
||||
const agentMap = new Map(params.agents.map((agent) => [agent.agentId, agent]));
|
||||
const agentMap = new Map(
|
||||
params.agents.map((agent) => [agent.agentId, agent]),
|
||||
);
|
||||
const cleaningCues = [...next.cleaningCues];
|
||||
for (const agentId of triggeredAgentIds) {
|
||||
const agent = agentMap.get(agentId);
|
||||
@@ -1248,7 +1301,8 @@ export const clearOfficeAnimationTriggerHold = (params: {
|
||||
};
|
||||
}
|
||||
if (params.hold === "call") {
|
||||
const directiveKey = next.phoneCallDirectiveKeyByAgentId[params.agentId] ?? "";
|
||||
const directiveKey =
|
||||
next.phoneCallDirectiveKeyByAgentId[params.agentId] ?? "";
|
||||
const phoneCallByAgentId = { ...next.phoneCallByAgentId };
|
||||
delete phoneCallByAgentId[params.agentId];
|
||||
return {
|
||||
@@ -1263,7 +1317,8 @@ export const clearOfficeAnimationTriggerHold = (params: {
|
||||
};
|
||||
}
|
||||
if (params.hold === "text") {
|
||||
const directiveKey = next.textMessageDirectiveKeyByAgentId[params.agentId] ?? "";
|
||||
const directiveKey =
|
||||
next.textMessageDirectiveKeyByAgentId[params.agentId] ?? "";
|
||||
const textMessageByAgentId = { ...next.textMessageByAgentId };
|
||||
delete textMessageByAgentId[params.agentId];
|
||||
return {
|
||||
@@ -1305,6 +1360,7 @@ export const buildOfficeAnimationState = (params: {
|
||||
const awaitingApprovalByAgentId: BooleanByAgentId = {};
|
||||
const deskHoldByAgentId: BooleanByAgentId = {};
|
||||
const gymHoldByAgentId: BooleanByAgentId = {};
|
||||
const jukeboxHoldByAgentId: BooleanByAgentId = {};
|
||||
const phoneBoothHoldByAgentId: BooleanByAgentId = {};
|
||||
const phoneCallByAgentId: PhoneCallByAgentId = {};
|
||||
const smsBoothHoldByAgentId: BooleanByAgentId = {};
|
||||
@@ -1353,9 +1409,11 @@ export const buildOfficeAnimationState = (params: {
|
||||
return {
|
||||
awaitingApprovalByAgentId,
|
||||
cleaningCues: params.state.cleaningCues,
|
||||
danceUntilByAgentId: {},
|
||||
deskHoldByAgentId,
|
||||
githubHoldByAgentId: params.state.githubHoldByAgentId,
|
||||
gymHoldByAgentId,
|
||||
jukeboxHoldByAgentId,
|
||||
manualGymUntilByAgentId: params.state.manualGymUntilByAgentId,
|
||||
pendingStandupRequest: params.state.pendingStandupRequest,
|
||||
phoneBoothHoldByAgentId,
|
||||
|
||||
@@ -3,17 +3,20 @@ export const OFFICE_INTERACTION_TARGETS = [
|
||||
"server_room",
|
||||
"meeting_room",
|
||||
"gym",
|
||||
"jukebox",
|
||||
"qa_lab",
|
||||
"sms_booth",
|
||||
"phone_booth",
|
||||
] as const;
|
||||
|
||||
export type OfficeInteractionTargetId = (typeof OFFICE_INTERACTION_TARGETS)[number];
|
||||
export type OfficeInteractionTargetId =
|
||||
(typeof OFFICE_INTERACTION_TARGETS)[number];
|
||||
|
||||
export const OFFICE_SKILL_TRIGGER_MOVEMENT_TARGETS = [
|
||||
"desk",
|
||||
"github",
|
||||
"gym",
|
||||
"jukebox",
|
||||
"qa_lab",
|
||||
] as const;
|
||||
|
||||
@@ -24,6 +27,7 @@ type OfficeSkillTriggerAnimationHoldKey =
|
||||
| "deskHoldByAgentId"
|
||||
| "githubHoldByAgentId"
|
||||
| "gymHoldByAgentId"
|
||||
| "jukeboxHoldByAgentId"
|
||||
| "qaHoldByAgentId";
|
||||
|
||||
export const OFFICE_SKILL_TRIGGER_PLACE_REGISTRY: Record<
|
||||
@@ -51,6 +55,11 @@ export const OFFICE_SKILL_TRIGGER_PLACE_REGISTRY: Record<
|
||||
animationHoldKey: "gymHoldByAgentId",
|
||||
alsoSetsSkillGymHold: true,
|
||||
},
|
||||
jukebox: {
|
||||
label: "Jukebox",
|
||||
interactionTarget: "jukebox",
|
||||
animationHoldKey: "jukeboxHoldByAgentId",
|
||||
},
|
||||
qa_lab: {
|
||||
label: "QA Lab",
|
||||
interactionTarget: "qa_lab",
|
||||
@@ -85,6 +94,20 @@ export const DEFAULT_SKILL_TRIGGER_FALLBACKS_BY_SKILL_KEY: Record<
|
||||
movementTarget: "desk",
|
||||
skipIfAlreadyThere: true,
|
||||
},
|
||||
soundclaw: {
|
||||
anyPhrases: [
|
||||
"spotify",
|
||||
"play a song",
|
||||
"play this song",
|
||||
"play music",
|
||||
"play a playlist",
|
||||
"find a song",
|
||||
"queue this song",
|
||||
"music link",
|
||||
],
|
||||
movementTarget: "jukebox",
|
||||
skipIfAlreadyThere: true,
|
||||
},
|
||||
};
|
||||
|
||||
export const buildOfficeSkillTriggerHoldMaps = (
|
||||
@@ -93,6 +116,7 @@ export const buildOfficeSkillTriggerHoldMaps = (
|
||||
deskHoldByAgentId: Record<string, boolean>;
|
||||
githubHoldByAgentId: Record<string, boolean>;
|
||||
gymHoldByAgentId: Record<string, boolean>;
|
||||
jukeboxHoldByAgentId: Record<string, boolean>;
|
||||
qaHoldByAgentId: Record<string, boolean>;
|
||||
skillGymHoldByAgentId: Record<string, boolean>;
|
||||
} => {
|
||||
@@ -100,6 +124,7 @@ export const buildOfficeSkillTriggerHoldMaps = (
|
||||
deskHoldByAgentId: {} as Record<string, boolean>,
|
||||
githubHoldByAgentId: {} as Record<string, boolean>,
|
||||
gymHoldByAgentId: {} as Record<string, boolean>,
|
||||
jukeboxHoldByAgentId: {} as Record<string, boolean>,
|
||||
qaHoldByAgentId: {} as Record<string, boolean>,
|
||||
skillGymHoldByAgentId: {} as Record<string, boolean>,
|
||||
};
|
||||
|
||||
+30
-10
@@ -1,6 +1,9 @@
|
||||
import type { RemovableSkillSource, SkillStatusEntry } from "@/lib/skills/types";
|
||||
import type {
|
||||
RemovableSkillSource,
|
||||
SkillStatusEntry,
|
||||
} from "@/lib/skills/types";
|
||||
|
||||
export type PackagedSkillId = "todo-board";
|
||||
export type PackagedSkillId = "soundclaw" | "todo-board";
|
||||
|
||||
export type PackagedSkillDefinition = {
|
||||
packageId: PackagedSkillId;
|
||||
@@ -30,20 +33,35 @@ const PACKAGED_SKILLS: PackagedSkillDefinition[] = [
|
||||
creatorName: "iamlukethedev",
|
||||
creatorUrl: "http://x.com/iamlukethedev/",
|
||||
},
|
||||
{
|
||||
packageId: "soundclaw",
|
||||
skillKey: "soundclaw",
|
||||
name: "soundclaw",
|
||||
description: "Control Spotify playback, search music, and return shareable music links.",
|
||||
installSource: "openclaw-workspace",
|
||||
creatorName: "iamlukethedev",
|
||||
creatorUrl: "https://github.com/iamlukethedev",
|
||||
},
|
||||
];
|
||||
|
||||
export const listPackagedSkills = (): PackagedSkillDefinition[] => [...PACKAGED_SKILLS];
|
||||
export const listPackagedSkills = (): PackagedSkillDefinition[] => [
|
||||
...PACKAGED_SKILLS,
|
||||
];
|
||||
|
||||
export const getPackagedSkillById = (packageId: string): PackagedSkillDefinition | null =>
|
||||
export const getPackagedSkillById = (
|
||||
packageId: string,
|
||||
): PackagedSkillDefinition | null =>
|
||||
PACKAGED_SKILLS.find((skill) => skill.packageId === packageId) ?? null;
|
||||
|
||||
export const getPackagedSkillBySkillKey = (skillKey: string): PackagedSkillDefinition | null => {
|
||||
export const getPackagedSkillBySkillKey = (
|
||||
skillKey: string,
|
||||
): PackagedSkillDefinition | null => {
|
||||
const normalized = skillKey.trim();
|
||||
return PACKAGED_SKILLS.find((skill) => skill.skillKey === normalized) ?? null;
|
||||
};
|
||||
|
||||
export const buildPackagedSkillStatusEntry = (
|
||||
skill: PackagedSkillDefinition
|
||||
skill: PackagedSkillDefinition,
|
||||
): SkillStatusEntry => ({
|
||||
name: skill.name,
|
||||
description: skill.description,
|
||||
@@ -62,11 +80,13 @@ export const buildPackagedSkillStatusEntry = (
|
||||
install: [],
|
||||
});
|
||||
|
||||
export const appendPackagedSkillsToMarketplace = (skills: SkillStatusEntry[]): SkillStatusEntry[] => {
|
||||
export const appendPackagedSkillsToMarketplace = (
|
||||
skills: SkillStatusEntry[],
|
||||
): SkillStatusEntry[] => {
|
||||
const presentKeys = new Set(skills.map((skill) => skill.skillKey.trim()));
|
||||
const additions = PACKAGED_SKILLS.filter((skill) => !presentKeys.has(skill.skillKey)).map(
|
||||
buildPackagedSkillStatusEntry
|
||||
);
|
||||
const additions = PACKAGED_SKILLS.filter(
|
||||
(skill) => !presentKeys.has(skill.skillKey),
|
||||
).map(buildPackagedSkillStatusEntry);
|
||||
if (additions.length === 0) {
|
||||
return skills;
|
||||
}
|
||||
|
||||
@@ -42,11 +42,18 @@ export type SkillMarketplaceEntry = {
|
||||
missingDetails: string[];
|
||||
};
|
||||
|
||||
const SKILL_MARKETPLACE_OVERRIDES: Record<string, Partial<SkillMarketplaceMetadata>> = {
|
||||
const SKILL_MARKETPLACE_OVERRIDES: Record<
|
||||
string,
|
||||
Partial<SkillMarketplaceMetadata>
|
||||
> = {
|
||||
github: {
|
||||
category: "Engineering",
|
||||
tagline: "Turns repository operations into a one-step teammate workflow.",
|
||||
capabilities: ["Pull request support", "Issue context", "Repository operations"],
|
||||
capabilities: [
|
||||
"Pull request support",
|
||||
"Issue context",
|
||||
"Repository operations",
|
||||
],
|
||||
featured: true,
|
||||
editorBadge: "Popular",
|
||||
rating: 4.9,
|
||||
@@ -64,14 +71,19 @@ const SKILL_MARKETPLACE_OVERRIDES: Record<string, Partial<SkillMarketplaceMetada
|
||||
slack: {
|
||||
category: "Communication",
|
||||
tagline: "Keeps agents plugged into team channels and notifications.",
|
||||
capabilities: ["Channel updates", "Message drafting", "Notification routing"],
|
||||
capabilities: [
|
||||
"Channel updates",
|
||||
"Message drafting",
|
||||
"Notification routing",
|
||||
],
|
||||
featured: true,
|
||||
rating: 4.7,
|
||||
installs: 14110,
|
||||
},
|
||||
linear: {
|
||||
category: "Planning",
|
||||
tagline: "Brings issue tracking and execution loops directly into agent workflows.",
|
||||
tagline:
|
||||
"Brings issue tracking and execution loops directly into agent workflows.",
|
||||
capabilities: ["Issue lookup", "Status updates", "Planning workflows"],
|
||||
featured: true,
|
||||
rating: 4.7,
|
||||
@@ -79,12 +91,26 @@ const SKILL_MARKETPLACE_OVERRIDES: Record<string, Partial<SkillMarketplaceMetada
|
||||
},
|
||||
"todo-board": {
|
||||
category: "Productivity",
|
||||
tagline: "Gives agents a shared workspace TODO board with blocked-task tracking.",
|
||||
capabilities: ["Task capture", "Blocked tracking", "Shared workspace state"],
|
||||
tagline:
|
||||
"Gives agents a shared workspace TODO board with blocked-task tracking.",
|
||||
capabilities: [
|
||||
"Task capture",
|
||||
"Blocked tracking",
|
||||
"Shared workspace state",
|
||||
],
|
||||
featured: true,
|
||||
editorBadge: "Claw3D test",
|
||||
hideStats: true,
|
||||
},
|
||||
soundclaw: {
|
||||
category: "Audio",
|
||||
tagline:
|
||||
"Lets agents search Spotify, control playback, and return music links on the current channel.",
|
||||
capabilities: ["Spotify search", "Playback control", "Same-channel link sharing"],
|
||||
featured: true,
|
||||
editorBadge: "Office demo",
|
||||
hideStats: true,
|
||||
},
|
||||
};
|
||||
|
||||
const hashString = (value: string): number => {
|
||||
@@ -122,7 +148,9 @@ const buildFallbackCapabilities = (skill: SkillStatusEntry): string[] => {
|
||||
return capabilities.slice(0, 3);
|
||||
};
|
||||
|
||||
const buildFallbackMetadata = (skill: SkillStatusEntry): SkillMarketplaceMetadata => {
|
||||
const buildFallbackMetadata = (
|
||||
skill: SkillStatusEntry,
|
||||
): SkillMarketplaceMetadata => {
|
||||
const normalizedKey = skill.skillKey.trim().toLowerCase();
|
||||
const source = skill.source.trim();
|
||||
const seed = hashString(`${normalizedKey}:${source}`);
|
||||
@@ -146,7 +174,9 @@ const buildFallbackMetadata = (skill: SkillStatusEntry): SkillMarketplaceMetadat
|
||||
: "Community";
|
||||
return {
|
||||
category,
|
||||
tagline: skill.description.trim() || `${titleCaseWords(skill.name)} capability pack.`,
|
||||
tagline:
|
||||
skill.description.trim() ||
|
||||
`${titleCaseWords(skill.name)} capability pack.`,
|
||||
trustLabel,
|
||||
capabilities: buildFallbackCapabilities(skill),
|
||||
featured: skill.bundled || source === "openclaw-managed",
|
||||
@@ -155,7 +185,9 @@ const buildFallbackMetadata = (skill: SkillStatusEntry): SkillMarketplaceMetadat
|
||||
};
|
||||
};
|
||||
|
||||
export const resolveSkillMarketplaceMetadata = (skill: SkillStatusEntry): SkillMarketplaceMetadata => {
|
||||
export const resolveSkillMarketplaceMetadata = (
|
||||
skill: SkillStatusEntry,
|
||||
): SkillMarketplaceMetadata => {
|
||||
const normalizedKey = skill.skillKey.trim().toLowerCase();
|
||||
const fallback = buildFallbackMetadata(skill);
|
||||
const override = SKILL_MARKETPLACE_OVERRIDES[normalizedKey];
|
||||
@@ -178,11 +210,15 @@ export const resolveSkillMarketplaceMetadata = (skill: SkillStatusEntry): SkillM
|
||||
};
|
||||
};
|
||||
|
||||
export const buildSkillMarketplaceEntry = (skill: SkillStatusEntry): SkillMarketplaceEntry => {
|
||||
export const buildSkillMarketplaceEntry = (
|
||||
skill: SkillStatusEntry,
|
||||
): SkillMarketplaceEntry => {
|
||||
const packagedSkill = getPackagedSkillBySkillKey(skill.skillKey);
|
||||
const missingDetails = buildSkillMissingDetails(skill);
|
||||
if (packagedSkill && !skill.baseDir.trim()) {
|
||||
missingDetails.unshift("Install this packaged Claw3D skill to make it available on the gateway.");
|
||||
missingDetails.unshift(
|
||||
"Install this packaged Claw3D skill to make it available on the gateway.",
|
||||
);
|
||||
}
|
||||
return {
|
||||
skill,
|
||||
@@ -195,7 +231,7 @@ export const buildSkillMarketplaceEntry = (skill: SkillStatusEntry): SkillMarket
|
||||
};
|
||||
|
||||
export const buildSkillMarketplaceCollections = (
|
||||
skills: SkillStatusEntry[]
|
||||
skills: SkillStatusEntry[],
|
||||
): Array<{
|
||||
id: SkillMarketplaceCollectionId;
|
||||
label: string;
|
||||
@@ -209,24 +245,40 @@ export const buildSkillMarketplaceCollections = (
|
||||
entries: SkillMarketplaceEntry[];
|
||||
}> = [];
|
||||
|
||||
const featured = entries.filter((entry) => entry.metadata.featured).slice(0, 6);
|
||||
const featured = entries
|
||||
.filter((entry) => entry.metadata.featured)
|
||||
.slice(0, 6);
|
||||
if (featured.length > 0) {
|
||||
collections.push({ id: "featured", label: "Featured", entries: featured });
|
||||
}
|
||||
|
||||
const claw3d = entries.filter((entry) => getPackagedSkillBySkillKey(entry.skill.skillKey));
|
||||
const claw3d = entries.filter((entry) =>
|
||||
getPackagedSkillBySkillKey(entry.skill.skillKey),
|
||||
);
|
||||
if (claw3d.length > 0) {
|
||||
collections.push({ id: "claw3d", label: "Claw3D", entries: claw3d });
|
||||
}
|
||||
|
||||
const installed = entries.filter((entry) => entry.readiness === "ready" || entry.skill.disabled);
|
||||
const installed = entries.filter(
|
||||
(entry) => entry.readiness === "ready" || entry.skill.disabled,
|
||||
);
|
||||
if (installed.length > 0) {
|
||||
collections.push({ id: "installed", label: "Installed", entries: installed });
|
||||
collections.push({
|
||||
id: "installed",
|
||||
label: "Installed",
|
||||
entries: installed,
|
||||
});
|
||||
}
|
||||
|
||||
const setupRequired = entries.filter((entry) => entry.readiness === "needs-setup");
|
||||
const setupRequired = entries.filter(
|
||||
(entry) => entry.readiness === "needs-setup",
|
||||
);
|
||||
if (setupRequired.length > 0) {
|
||||
collections.push({ id: "setup-required", label: "Needs setup", entries: setupRequired });
|
||||
collections.push({
|
||||
id: "setup-required",
|
||||
label: "Needs setup",
|
||||
entries: setupRequired,
|
||||
});
|
||||
}
|
||||
|
||||
for (const group of sourceGroups) {
|
||||
|
||||
@@ -140,6 +140,53 @@ const TODO_BOARD_EXAMPLE_JSON = `{
|
||||
}
|
||||
`;
|
||||
|
||||
// Keep this string synchronized with assets/skills/soundclaw/SKILL.md.
|
||||
const SOUNDCLAW_SKILL_MD = `---
|
||||
name: soundclaw
|
||||
description: Control Spotify playback, search music, and return shareable music links.
|
||||
metadata: {"openclaw":{"skillKey":"soundclaw"}}
|
||||
---
|
||||
|
||||
# SOUNDCLAW
|
||||
|
||||
Use this skill when the user wants an agent to search for music, play a song or playlist, control Spotify playback, or send back a shareable Spotify link on the same channel the request came from.
|
||||
|
||||
## Trigger
|
||||
|
||||
\`\`\`json
|
||||
{
|
||||
"activation": {
|
||||
"anyPhrases": [
|
||||
"spotify",
|
||||
"play a song",
|
||||
"play this song",
|
||||
"play music",
|
||||
"play a playlist",
|
||||
"find a song",
|
||||
"queue this song",
|
||||
"music link"
|
||||
]
|
||||
},
|
||||
"movement": {
|
||||
"target": "jukebox",
|
||||
"skipIfAlreadyThere": true
|
||||
}
|
||||
}
|
||||
\`\`\`
|
||||
|
||||
When this skill is activated, the agent should walk to the office jukebox before handling the request.
|
||||
|
||||
- Treat requests from Telegram or any other external surface as valid triggers when they ask for Spotify playback, search, queueing, or music-link sharing.
|
||||
- The physical behavior for this skill is: go to the jukebox, perform the music-selection workflow, then report the result.
|
||||
- If the agent is already at the jukebox, continue without adding extra movement narration.
|
||||
|
||||
## Channel behavior
|
||||
|
||||
- Reply on the same active channel or session that received the request.
|
||||
- If playback cannot start but a matching track, album, or playlist is found, send back the best Spotify link instead of failing silently.
|
||||
- If multiple matches are plausible, ask a clarifying question instead of guessing.
|
||||
`;
|
||||
|
||||
const PACKAGED_SKILL_FILES: Record<string, PackagedSkillFile[]> = {
|
||||
"todo-board": [
|
||||
{
|
||||
@@ -151,9 +198,17 @@ const PACKAGED_SKILL_FILES: Record<string, PackagedSkillFile[]> = {
|
||||
content: TODO_BOARD_EXAMPLE_JSON,
|
||||
},
|
||||
],
|
||||
soundclaw: [
|
||||
{
|
||||
relativePath: "SKILL.md",
|
||||
content: SOUNDCLAW_SKILL_MD,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
export const readPackagedSkillFiles = (packageId: string): PackagedSkillFile[] => {
|
||||
export const readPackagedSkillFiles = (
|
||||
packageId: string,
|
||||
): PackagedSkillFile[] => {
|
||||
const files = PACKAGED_SKILL_FILES[packageId];
|
||||
if (!files || files.length === 0) {
|
||||
throw new Error(`Packaged skill assets are missing: ${packageId}`);
|
||||
|
||||
@@ -54,9 +54,16 @@ describe("skill triggers", () => {
|
||||
});
|
||||
|
||||
it("keeps trigger places and fallback definitions in one central registry", () => {
|
||||
expect(OFFICE_SKILL_TRIGGER_PLACE_REGISTRY.desk.interactionTarget).toBe("desk");
|
||||
expect(OFFICE_SKILL_TRIGGER_PLACE_REGISTRY.github.interactionTarget).toBe("server_room");
|
||||
expect(DEFAULT_SKILL_TRIGGER_FALLBACKS_BY_SKILL_KEY["todo-board"]?.movementTarget).toBe("desk");
|
||||
expect(OFFICE_SKILL_TRIGGER_PLACE_REGISTRY.desk.interactionTarget).toBe(
|
||||
"desk",
|
||||
);
|
||||
expect(OFFICE_SKILL_TRIGGER_PLACE_REGISTRY.github.interactionTarget).toBe(
|
||||
"server_room",
|
||||
);
|
||||
expect(
|
||||
DEFAULT_SKILL_TRIGGER_FALLBACKS_BY_SKILL_KEY["todo-board"]
|
||||
?.movementTarget,
|
||||
).toBe("desk");
|
||||
});
|
||||
|
||||
it("builds animation hold maps from the central place registry", () => {
|
||||
|
||||
Reference in New Issue
Block a user