Add ignore_class parameter to SparseCategoricalCrossentropy

[cpuguy96]

Introduced ignore_class option to SparseCategoricalCrossentropy to allow ignoring specific classes during loss computation. Added unit tests to validate functionality of ignore_class with both standard and weighted inputs, as well as with logits.

[gemini-code-assist]

Summary of Changes

Hello @cpuguy96, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the SparseCategoricalCrossentropy metric by providing a new parameter to ignore a specified class during loss calculation. This feature is particularly useful in scenarios where certain classes should not contribute to the loss, such as handling padding tokens or uninteresting background classes, thereby improving model training flexibility and accuracy in specific contexts.

Highlights

• New Feature: Introduced an ignore_class option to the SparseCategoricalCrossentropy metric, allowing specific classes to be excluded from loss computation.
• Testing: Added comprehensive unit tests to validate the ignore_class functionality, covering standard inputs, weighted inputs, and logits.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• keras/src/metrics/probabilistic_metrics.py
  • Added ignore_class parameter to the SparseCategoricalCrossentropy class constructor.
  • Initialized and stored the ignore_class value within the metric instance.
  • Included ignore_class in the get_config method for proper serialization.
• keras/src/metrics/probabilistic_metrics_test.py
  • Added test_ignore_class to verify the basic functionality of ignoring a class.
  • Added test_ignore_class_weighted to ensure correct behavior when ignore_class is used with sample weights.
  • Added test_ignore_class_from_logits to validate ignore_class when inputs are provided as logits.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[gemini-code-assist]

Code Review

This pull request introduces the ignore_class parameter to the SparseCategoricalCrossentropy metric, aligning its functionality with the corresponding loss function. The changes correctly plumb the new parameter through the class constructor and serialization logic. Comprehensive unit tests have been added to validate the new functionality under various conditions, including with sample weights and logits. The implementation is sound and consistent with the existing codebase.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.88%. Comparing base (5f803fd) to head (0e8d2df).
⚠️ Report is 63 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #22284   +/-   ##
=======================================
  Coverage   82.87%   82.88%           
=======================================
  Files         594      594           
  Lines       65798    65799    +1     
  Branches    10288    10288           
=======================================
+ Hits        54533    54536    +3     
+ Misses       8643     8642    -1     
+ Partials     2622     2621    -1     

Flag	Coverage Δ
keras	82.70% <100.00%> (+<0.01%)	⬆️
keras-jax	60.92% <100.00%> (+<0.01%)	⬆️
keras-numpy	55.07% <100.00%> (+<0.01%)	⬆️
keras-openvino	49.06% <33.33%> (-0.01%)	⬇️
keras-tensorflow	62.12% <100.00%> (+<0.01%)	⬆️
keras-torch	60.99% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[hertschuh]

Thanks for filling this gap!

[hertschuh]

This should not be needed, please revert.

And if it is needed, this is an indication that there is a different bug.

[cpuguy96]

This was added due to test failures in the new tests added only for torch.

FAILED keras/src/metrics/probabilistic_metrics_test.py::SparseCategoricalCrossentropyTest::test_ignore_class - TypeError: unsupported operand type(s) for *: 'numpy.ndarray' and 'Tensor'
FAILED keras/src/metrics/probabilistic_metrics_test.py::SparseCategoricalCrossentropyTest::test_ignore_class_from_logits - TypeError: unsupported operand type(s) for *: 'numpy.ndarray' and 'Tensor'
FAILED keras/src/metrics/probabilistic_metrics_test.py::SparseCategoricalCrossentropyTest::test_ignore_class_weighted - TypeError: unsupported operand type(s) for *: 'numpy.ndarray' and 'Tensor'

I updated the function to use ops.multiply instead of * to remove the need of calling ops.convert_to_tensor.

[hertschuh]

Yes, that is the right fix.

[hertschuh]

Thanks!