chore(amber): drop duplicate praw from requirements.txt#5689
Open
Yicong-Huang wants to merge 1 commit into
Open
chore(amber): drop duplicate praw from requirements.txt#5689Yicong-Huang wants to merge 1 commit into
Yicong-Huang wants to merge 1 commit into
Conversation
praw==7.6.1 was pinned in both amber/requirements.txt and amber/operator-requirements.txt. praw is only used by the Reddit Search source operator, so it belongs in operator-requirements.txt. Drop the duplicate from requirements.txt; the operator pin (and lockfile) stay unchanged. Closes apache#5687
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5689 +/- ##
=========================================
Coverage 52.75% 52.75%
Complexity 2546 2546
=========================================
Files 1090 1090
Lines 42150 42150
Branches 4529 4529
=========================================
Hits 22238 22238
Misses 18589 18589
Partials 1323 1323
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 399 | 0.244 | 23,323/34,040/34,040 us | 🔴 +12.6% / ⚪ within ±5% |
| 🔴 | bs=100 sw=10 sl=64 | 820 | 0.5 | 121,520/143,789/143,789 us | 🔴 +17.7% / 🔴 +8.9% |
| 🔴 | bs=1000 sw=10 sl=64 | 930 | 0.568 | 1,073,987/1,118,744/1,118,744 us | 🔴 +20.3% / 🔴 -11.3% |
Baseline details
Latest main 4fd395b from 2026-06-13T21:15:43.559Z
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 399 tuples/sec | 425.86 tuples/sec | 411.23 tuples/sec | -6.3% | -3.0% |
| bs=10 sw=10 sl=64 | MB/s | 0.244 MB/s | 0.26 MB/s | 0.251 MB/s | -6.1% | -2.8% |
| bs=10 sw=10 sl=64 | p50 | 23,323 us | 24,156 us | 23,773 us | -3.4% | -1.9% |
| bs=10 sw=10 sl=64 | p95 | 34,040 us | 30,243 us | 35,177 us | +12.6% | -3.2% |
| bs=10 sw=10 sl=64 | p99 | 34,040 us | 30,243 us | 35,177 us | +12.6% | -3.2% |
| bs=100 sw=10 sl=64 | throughput | 820 tuples/sec | 957.92 tuples/sec | 896.95 tuples/sec | -14.4% | -8.6% |
| bs=100 sw=10 sl=64 | MB/s | 0.5 MB/s | 0.585 MB/s | 0.547 MB/s | -14.5% | -8.7% |
| bs=100 sw=10 sl=64 | p50 | 121,520 us | 103,271 us | 111,596 us | +17.7% | +8.9% |
| bs=100 sw=10 sl=64 | p95 | 143,789 us | 144,102 us | 139,415 us | -0.2% | +3.1% |
| bs=100 sw=10 sl=64 | p99 | 143,789 us | 144,102 us | 139,415 us | -0.2% | +3.1% |
| bs=1000 sw=10 sl=64 | throughput | 930 tuples/sec | 1,118 tuples/sec | 1,048 tuples/sec | -16.8% | -11.3% |
| bs=1000 sw=10 sl=64 | MB/s | 0.568 MB/s | 0.682 MB/s | 0.64 MB/s | -16.7% | -11.2% |
| bs=1000 sw=10 sl=64 | p50 | 1,073,987 us | 892,725 us | 966,517 us | +20.3% | +11.1% |
| bs=1000 sw=10 sl=64 | p95 | 1,118,744 us | 943,636 us | 1,016,694 us | +18.6% | +10.0% |
| bs=1000 sw=10 sl=64 | p99 | 1,118,744 us | 943,636 us | 1,016,694 us | +18.6% | +10.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,500.96,200,128000,399,0.244,23323.40,34039.65,34039.65
1,100,10,64,20,2439.29,2000,1280000,820,0.500,121520.26,143789.08,143789.08
2,1000,10,64,20,21500.27,20000,12800000,930,0.568,1073987.13,1118743.79,1118743.79
Contributor
Author
|
cc @SarahAsad23 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
praw==7.6.1was pinned in two requirements files:amber/requirements.txtprawlineamber/operator-requirements.txtBoth files are installed sequentially by
bin/computing-unit-master.dockerfileandbin/computing-unit-worker.dockerfile, so removing the duplicate is a runtime no-op — pip resolves the single remaining pin fromoperator-requirements.txt. Lockfile (amber/system-requirements-lock.txt) andLICENSE-binary-pythonare untouched because praw is still pulled in by the operator file.Any related issues, documentation, discussions?
Closes #5687
(Supersedes #5688, which was wrongly scoped — that PR proposed to also delete the Reddit Search operator itself. Closed without merging.)
How was this PR tested?
git diff upstream/main— confirms only the singlepraw==7.6.1line is removed fromamber/requirements.txtgrep -rn "praw" amber/— confirms praw remains pinned inoperator-requirements.txt,system-requirements-lock.txt, andLICENSE-binary-pythonRedditSearchSourceOpDesc.scala), so the operator continues to work as before with the single remaining pinWas this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.7 [1M context])