Skip to content

Add precision-recall curves#67

Merged
ablaom merged 34 commits intodevfrom
pr-curve
Mar 7, 2026
Merged

Add precision-recall curves#67
ablaom merged 34 commits intodevfrom
pr-curve

Conversation

@ablaom
Copy link
Copy Markdown
Member

@ablaom ablaom commented Feb 2, 2026

Closes #25

Addition of AveragePrecision (a form of "area under the precision-recall curve") to be provided in a follow-up PR.

Implementation notes

  • As in sk-learn, I have added a more general core method Functions.confusion_counts_at_threshold (in sk-learn this is called confusion_matrix_at_thresholds) and have refactored the existing Functions.roc function to make use of that. The new Functions.precision_recall_curve method is then pretty trivial to implement. When you compute what you need for roc, you basically get all the confusion matrix entries along the way, so there's no real loss in performance for the refactored roc.

  • There's some considerable refactoring of document strings to lower maintenance burden, which is a little complicated, so reviewer should generate the Documenter.jl manual and review:

    • the new section "Precision Recall Curves"
    • the following docstrings: Functions.precision_recall_curve, Functions.roc, Functions.confusion_counts_at_thresholds, precision_recall_curve, roc.

    There were a number of mistakes in the roc docstrings around the threshold and associated bins, which hopefully I have correct now.

I have also fixed some incorrect logging tests that crept in a recent PR on the Boyce Index.

@ablaom
Copy link
Copy Markdown
Member Author

ablaom commented Feb 2, 2026

cc @DilumAluthge

This was referenced Feb 10, 2026
@ablaom
Copy link
Copy Markdown
Member Author

ablaom commented Mar 1, 2026

Super helpful review @OkonSamuel , thank you. Your suggestions will make understanding the code much better.

Unfortunately, I'm having trouble committing the suggestions in a batch. Any chance you have a branch that already includes them collectively? If not, let me know and I'll manually commit them.

@OkonSamuel
Copy link
Copy Markdown
Member

Super helpful review @OkonSamuel , thank you. Your suggestions will make understanding the code much better.

Unfortunately, I'm having trouble committing the suggestions in a batch. Any chance you have a branch that already includes them collectively? If not, let me know and I'll manually commit them.

Hi Anthony, I'm sorry I didn't commit them in a branch. You can manually commit them and I can go through it once more before you merge it in.

@ablaom
Copy link
Copy Markdown
Member Author

ablaom commented Mar 5, 2026

@OkonSamuel I was able to commit your suggestions by doing them one at a time in reverse order. There was one instance where I think you inadvertently deleted a little too much, which I have restored, namely lines 76-80 of src/functions.jl.

I have reviewed the rendering of the docs and it looks good. Fixed a couple of broken links that I discovered.

Ready for your second look.

Copy link
Copy Markdown
Member

@OkonSamuel OkonSamuel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!!

@ablaom ablaom merged commit be4cf9c into dev Mar 7, 2026
3 checks passed
This was referenced Mar 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: pr_curve convenience function to generate the precision-recall curve (PR curve)

2 participants