feat(mcp): Add MCP servers for Babylon.js graph editors#18344
feat(mcp): Add MCP servers for Babylon.js graph editors#18344RaananW wants to merge 15 commits intoBabylonJS:masterfrom
Conversation
Add Model Context Protocol (MCP) servers that expose Babylon.js authoring workflows to MCP-compatible clients (VS Code Copilot, Claude, etc.). Servers included: - mcp-server-core: shared helpers for validation, response shaping, schemas - nme-mcp-server: Node Material Editor graph authoring with live session bridge - nge-mcp-server: Node Geometry Editor graph authoring - npe-mcp-server: Node Particle Editor graph authoring - nrge-mcp-server: Node Render Graph Editor graph authoring - flow-graph-mcp-server: Flow Graph visual scripting - gui-mcp-server: Babylon.js GUI layout authoring - gltf-mcp-server: glTF/glb asset authoring and validation - smart-filters-mcp-server: Smart Filters post-processing chains Each server includes README, unit tests, and example outputs. Also adds MCP session UI components to the Node Editor and Playground.
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Building or testing the playground has failed. If the tests failed, results can be found here: |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18344/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18344/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18344/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
Smart Filters Editor is available to test at: |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18344/merge/ |
- Change mcp-server-core imports from dist/index.js to src/index.js so vitest can resolve TypeScript source directly without a build step - Add babylonjs-gltf2interface stub alias to vitest.config.mts (matching master) so glTF loader tests can resolve the types-only package
- Replace 20 barrel (directory) imports with direct file imports - Rename type params to StrictPascalCase (TSchema→Schema, etc.) - Rename IJsonObject→JsonObject, _parseJsonObject→ParseJsonObject - Add Async suffix: resolveExternalBuffers→resolveExternalBuffersAsync - Rename _getNumComponents→_GetNumComponents (static private) - Add missing JSDoc @param/@returns annotations - Replace Promise.then with async IIFE in signal handlers - Merge duplicate core/Misc/gradients imports
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Reviewer - this PR has made changes to one or more package.json files. |
- Fix className casing to match engine RegisterClass names: Csm→Cascaded, Fxaa→FXAA, Ssao2→SSAO2, Ssr→SSR, Taa→TAA - Add missing blocks: ComputeShaderBlock, IblShadowsRendererBlock, PassCubePostProcessBlock - Update documentation strings with corrected block names
|
Building or testing the playground has failed. If the tests failed, results can be found here: |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18344/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18344/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18344/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
Smart Filters Editor is available to test at: |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18344/merge/ |
Flow Graph: add pointer events (Down/Up/Move), physics collision event, audio sound ended event, physics execution blocks (ApplyForce, ApplyImpulse, SetLinearVelocity, SetAngularVelocity, SetMotionType), physics data blocks (GetLinearVelocity, GetAngularVelocity, GetMassProperties), audio execution blocks (PlaySound, StopSound, PauseSound, SetVolume), audio data blocks (GetVolume, IsSoundPlaying), matrix combine/extract blocks (Matrix2D, Matrix3D), and DebugBlock. Remove stale 'NOT yet implemented' comments for blocks now in core. NGE: add ExtrudeGeometryBlock to registry.
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Building or testing the playground has failed. If the tests failed, results can be found here: |
|
Smart Filters Editor is available to test at: |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18344/merge/ |
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Smart Filters Editor is available to test at: |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18344/merge/ |
|
Building or testing the playground has failed. If the tests failed, results can be found here: |
|
Building or testing the playground has failed. If the tests failed, results can be found here: |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
…oaders These files were inadvertently touched by the workspace-wide lint fix. Revert them to master to keep the PR scoped to MCP server changes only.
Keep MCP config local — don't impose it on all contributors.
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Smart Filters Editor is available to test at: |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Smart Filters Editor is available to test at: |
|
You have changed file(s) that made possible changes to the sandbox. https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18344/merge/ |
|
Building or testing the playground has failed. If the tests failed, results can be found here: |
|
Reviewer - this PR has made changes to one or more package.json files. |
|
Smart Filters Editor is available to test at: |
3 participants
Summary
Add Model Context Protocol (MCP) servers that expose Babylon.js authoring workflows to MCP-compatible clients (VS Code Copilot, Claude, etc.). Each server manages one Babylon.js subsystem in memory, allowing AI assistants to create, edit, validate, and export graphs through structured tool calls.
Servers
mcp-server-corenme-mcp-servernge-mcp-servernpe-mcp-servernrge-mcp-serverflow-graph-mcp-servergui-mcp-servergltf-mcp-serversmart-filters-mcp-serverArchitecture
@tools/*workspace packages underpackages/tools/rollup.config.mcp.mjs, producing standalonedist/index.jswith shebang@modelcontextprotocol/sdkandzod/v4for schema validationmcp-server-coreAlso included
.vscode/mcp.jsonworkspace configuration for local testingvitest.config.mtsalias path fix for test resolutionTesting
All MCP-specific tests pass (600+ tests across 31 test files). One pre-existing failure in
nmeParse.test.tsis caused by a new core shader import (gaussianSplattingVoxel) on master that breaks the deep import chain in vitest — not related to this PR.Future work
A Scene MCP server (orchestration layer that consumes exported JSON from these servers to produce runnable scenes) will follow in a separate PR once this lands.