Add support for MessageGroupId in standard queues#2442
Add support for MessageGroupId in standard queues#2442areinel wants to merge 5 commits intocelery:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds support for MessageGroupId in standard SQS queues to enable AWS's fair queues feature, which helps mitigate "noisy neighbor" issues in multi-tenant environments.
Changes:
- Simplified FIFO queue MessageGroupId handling to use
.get()with a default value - Added support for passing MessageGroupId to standard (non-FIFO) queues
- Code formatting improvements for better readability
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if 'MessageGroupId' in message['properties']: | ||
| kwargs['MessageGroupId'] = message['properties']['MessageGroupId'] |
There was a problem hiding this comment.
The new functionality to support MessageGroupId in standard queues lacks test coverage. The existing test_predefined_queues_put_to_queue only verifies DelaySeconds behavior. Consider adding a test case that verifies MessageGroupId is correctly passed to send_message for standard queues when provided in message properties.
Co-authored-by: Copilot <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2442 +/- ##
==========================================
+ Coverage 81.19% 81.24% +0.04%
==========================================
Files 77 77
Lines 9738 9741 +3
Branches 1098 1099 +1
==========================================
+ Hits 7907 7914 +7
+ Misses 1625 1623 -2
+ Partials 206 204 -2 ☔ View full report in Codecov by Sentry. |
AWS now supports MessageGroupId on standard queues to enable fair queues, which mitigate the “noisy neighbor” impact in multi-tenant queues and help maintain consistent message dwell times across tenants
(https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fair-queues.html).
This change ensures:
FIFO queues always get MessageGroupId (defaulting to default) and MessageDeduplicationId.
Standard queues can pass MessageGroupId without triggering FIFO-only logic.
Note: reopened PR from branch instead of main**