Parallelize command buffer recording

[IAmNotHanni]

Is your feature request related to a problem?

Command buffers should be recorded in parallel to make sure we finish on the cpu side as fast as possible to start rendering a frame in the gpu.

Description

In theory, the command buffer of each graphics pass can be recorded in parallel using taskflow. There is a tradeoff to be made when it comes to command buffer recording, because on the one side we want to keep the number of command buffers to a minimum, but we also need to make sure the work is balanced more or less evenly. There are several options:

• Record everything into one command buffer (every pass of every rendermodule): This is not optimal, because it is not parallel at all.
• Record command buffers on a per-rendermodule basis: This is slightly better, but not fine-grained enough
• Record command buffers on a per-pass basis for each rendermodule: This seems to be the optimal solution.

We should have one command pool per thread and queue and frame in flight, meaning if we have graphics, transfer, and compute queue, and we have let's say 10 threads and 2 frames in flight, we will have a total of 60 command pools, 6 command pools per thread. We can use thread_local to achieve per-thread command pools.

We should avoid VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT for this and use vkResetCommandPool for every frame we render.

Alternatives

If we keep it single-threaded, we waste a lot of performance, and the renderer is not scalable.

Affected Code

Rendergraph and command buffer / command pool wrapper code

Operating System

All

Additional Context

None