Batch Scheduling

[Jackmin801]

We can probably be smarter about the verification by batching common lengths together. Might not be worth it though because we would fail slower.

[Jackmin801]

Now that I've thought about it abit more and seen more stuff I feel like this batching might be better done in a scheduler. Similar to vLLM