Proposal: Rendering Window API for Custom Gizmos and Model Interaction

[Isaina]

Hello,

I would like to propose adding or exposing a Rendering Window API in Coin3D that would allow external developers to create custom interactive tools inside the 3D viewport.

The main goal would be to enable plugins or external modules to add:

• custom viewport gizmos,
• small floating command windows,
• interactive buttons inside the 3D view,
• object manipulation tools,
• keyboard-based movement and rotation,
• visual annotations,
• simple CAD-style interaction overlays.

For example, when a user selects a 3D object, a plugin could display a small custom interaction panel near the object or inside the viewport, with commands such as:

Move / Copy / Edit / Rotate - / Rotate + / OK / Cancel

This would make it possible to build more advanced CAD/BIM-style workflows on top of Coin 3D without modifying the core application.

Requested API capabilities

It would be very useful if Coin 3D exposed an API for:

1. Drawing custom 2D UI elements over the 3D viewport
2. Anchoring UI elements to selected 3D objects or screen positions
3. Receiving mouse click events on custom viewport buttons
4. Receiving keyboard events for object manipulation
5. Moving, rotating, or editing selected objects programmatically
6. Adding temporary measurement lines, labels, arrows, and annotations
7. Creating plugin-defined interaction modes
8. Refreshing or invalidating the viewport after external changes

Use case

One possible use case is a CAD/BIM plugin where the user selects a duct, pipe, wall, or mechanical component and then receives a small contextual gizmo directly inside the 3D view.

This would allow the user to move the object, rotate it, edit parameters, confirm the operation, or cancel it without opening separate dialogs.

The interaction could look similar to lightweight CAD viewport controls or Blender-style gizmos, but adapted to Coin 3D.

Why this would be useful

Such an API would make Coin 3D more extensible and allow developers to build:

• CAD tools,
• BIM tools,
• HVAC routing tools,
• object editing tools,
• measurement tools,
• annotation tools,
• visual scripting helpers,
• interactive engineering workflows.

This would also help keep the Coin 3D core clean, while allowing advanced functionality to be developed as plugins or external modules.

Suggested direction

A minimal first version could include:

• viewport overlay drawing,
• mouse event callbacks,
• keyboard event callbacks,
• selected object access,
• transform API for move/rotate/scale,
• simple text/rectangle/button drawing,
• viewport refresh function.

Later, this could evolve into a more complete plugin API for interactive 3D tools.

Thank you for considering this proposal.

[Isaina]

One more picture of suggestion

[VolkerEnderlein]

@veelo what do you think about this? To me it feels like a reimplementation of the ImGUI library. Mercury and Thermo Fisher OpenInventor implemented some UI elements.

[veelo]

I don't know the libraries that you mention, but I think this functionality is outside the scope of Coin. I think this is easier implemented as an overlay in some other library, like Qt.

[WizzerWorks]

I agree that this is out of scope for the coin core library. It could be implemented as an extension library of widgets and utilities.

[Isaina]

do you maybe have some help how to write plugin for coin core library some api to hook or there is a sample of others plugins

[veelo]

If you use QuarterWidget, I think you can derive from it and draw directly on it, over the Coin rendering, using the Qt draw API. We have derived from Quarter to extend some functionality and have implemented a context sensitive popup. You'll likely want to block events reaching Coin while the GUI is active. Using an existing GUI framework will probably get you there the fastest, you can have styling that matches your OS, get conveniences like font rendering, good icon support, tool tips etc., but it will of course only work with that exact framework.

If you want to be framework agnostic, I guess you can implement it using only Coin, using an orthographic camera and display the result over the 3D scene, a little bit like you would use SmAxisDisplayKit from https://github.com/coin3d/smallchange/tree/master/lib/SmallChange/nodekits. There are probably better examples that are a 2D, like https://github.com/coin3d/smallchange/blob/master/lib/SmallChange/nodekits/LegendKit.cpp, but I haven't used those. In this approach you might have to do more work, depending on what you want to achieve.

[veelo]

See also SoRenderManager::addSuperimposition().