Set Model Temperature to 0 for Consistent Leaderboard Results

[HuanzhiMao]

The current model generation script (model_handlers) uses a default temperature of 0.7 for inference. This introduces some degree of randomness into the model output generation, leading to potential variability in the evaluation scores from run to run.
For benchmarking purposes, we should set it to 0 for consistency and reliability of the evaluation results.

[ShishirPatil]

What do folks thinks about this? I'm mostly ok as long as we are consistent across all models.

[alonsosilvaallende]

I fully agree with a much lower temperature. Temperature 0.7 is extremely high. However, if I remember correctly it cannot be exactly zero for some models I have tried but strictly positive. Therefore, I think a positive but much smaller value would be better, for example Temperature=0.01.

[hexists]

hello.
I think lowering the temperature is a good idea.
I think it would be good to set the temperature to 0 or at least a low value close to 0 for reproducibility.
Thank you for providing the BFCL.

[ShishirPatil]

Thank you @aastroza and @hexists for weighing in. Ok @HuanzhiMao let's go with a lower temperature then maybe something like 0.1? But this would change fundamentally all numbers in the leaderboard. So, once we land all the existing PRs we can do this? I'll keep this issue open.

[HuanzhiMao]

Thank you @aastroza and @hexists for weighing in. Ok @HuanzhiMao let's go with a lower temperature then maybe something like 0.1? But this would change fundamentally all numbers in the leaderboard. So, once we land all the existing PRs we can do this? I'll keep this issue open.

Yea agree. Let's wait till all PR are merged and then we update this.

[ShishirPatil]

This has been merged, so should we close the discussion down?

[HuanzhiMao]

Temperature has been set to 0 in #574.