Render hangs when layout shows different scene

[p-weston]

1.0.0-rc.3

Render hangs when the layout for the scene being rendered shows a different scene on the canvas.

eg, in the attached file layout "Scene 0" is showing "Scene 1" on the canvas, so rendering "Scene 0" hangs until the render is cancelled.

I have a small fix (resetting the layout before rendering) Shall I submit a PR?

[rodlie]

Thanks for the report, I though this was "fixed" in 52409f6 (v0.9.6.1), been a while since I added this, so don't remember if this was good enough.

[p-weston]

I'm not 100% on how it should be working, but from the code, I think it switches to the layout with the right scene in fScene, which is the scene the layout was created with rather than the one currently showing on the canvas.

I put the reset after line 318 in that diff, as the current layout might need a reset.

[rodlie]

Can confirm the issue, it works fine on first render (why I didn't detect the issue), but the next time the RenderWidget::render function runs the handler->isCurrentScene(index) seems to be wrong.

[rodlie]

My proposed solution: bc6f8a5

This is of course a workaround until we detach the renderer from app, ~v1.2/v1.3.

[p-weston]

There is a good chance I'm misunderstanding something, but shouldn't the resetCurrentScene come after the setCurrent? reset the one we are changing to rather than the one we were on.

[rodlie]

SetCurrent will never run, as index == mCurrentId.

[rodlie]

There is something iffy with the layouthandler index, but that's probably a rabbit hole of tech debt from enve (that I will rewrite at some point).

[p-weston]

That's probably part of my confusion! That's fixed most cases, but I can still see the problem with that test file if I open the file and switch the current layout to "Scene 1" before rendering "Scene 0".
I think that is fixed by swapping those two lines.

[rodlie]

bc6f8a5 works as it should here, on my test project and yours.

select scene 1, press renderer and current scene is now 0, select scene 1, press renderer and current scene is now 0. Also tested with different scenes in queue.

Test binary: https://github.com/friction2d/friction/actions/runs/22263944453 (scroll down and download artifact).

[p-weston]

ah... I think I see. With your binary, and my test file, rendering "Scene 0" with the current layout set to "Scene 0" fixes the file.

In my test file both layout "Scene 0" and layout "Scene 1" have the canvas set to show scene "Scene 1". So opening the file, selecting layout "Scene 1", and then trying to render scene "Scene 0" hangs. I think what happens is:

1. reset happens to layout "Scene 1", setting its visible canvas to "Scene 1"
2. setCurrent swaps the current layout to layout "Scene 0" (as its fScene matches scene "Scene 0") which sets the visible canvas to "Scene 1"

The layout and scene names all matching make it very hard to describe!