Answer relevancy currently does not penalise claims with an 'idk' verdict.
You have recently introduced 'penalize_ambiguous_claims' as a parameter for faithfulness to allow the end user to determine whether 'idk' claims should penalise the score (with the default of False not penalising the score).
A similar feature for answer relevancy would be welcome.
Answer relevancy currently does not penalise claims with an 'idk' verdict.
You have recently introduced 'penalize_ambiguous_claims' as a parameter for faithfulness to allow the end user to determine whether 'idk' claims should penalise the score (with the default of False not penalising the score).
A similar feature for answer relevancy would be welcome.