Skip to content

chore(amber): drop duplicate praw from requirements.txt#5689

Open
Yicong-Huang wants to merge 1 commit into
apache:mainfrom
Yicong-Huang:chore/dedup-praw-from-requirements
Open

chore(amber): drop duplicate praw from requirements.txt#5689
Yicong-Huang wants to merge 1 commit into
apache:mainfrom
Yicong-Huang:chore/dedup-praw-from-requirements

Conversation

@Yicong-Huang

@Yicong-Huang Yicong-Huang commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

praw==7.6.1 was pinned in two requirements files:

File Role Status
amber/requirements.txt engine / SDK deps drop the duplicate praw line
amber/operator-requirements.txt operator-specific deps (where praw belongs — only the Reddit Search operator uses it) unchanged

Both files are installed sequentially by bin/computing-unit-master.dockerfile and bin/computing-unit-worker.dockerfile, so removing the duplicate is a runtime no-op — pip resolves the single remaining pin from operator-requirements.txt. Lockfile (amber/system-requirements-lock.txt) and LICENSE-binary-python are untouched because praw is still pulled in by the operator file.

Any related issues, documentation, discussions?

Closes #5687

(Supersedes #5688, which was wrongly scoped — that PR proposed to also delete the Reddit Search operator itself. Closed without merging.)

How was this PR tested?

  • git diff upstream/main — confirms only the single praw==7.6.1 line is removed from amber/requirements.txt
  • grep -rn "praw" amber/ — confirms praw remains pinned in operator-requirements.txt, system-requirements-lock.txt, and LICENSE-binary-python
  • No source code uses praw directly (only the Reddit Search operator's generated Python in RedditSearchSourceOpDesc.scala), so the operator continues to work as before with the single remaining pin

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.7 [1M context])

praw==7.6.1 was pinned in both amber/requirements.txt and
amber/operator-requirements.txt. praw is only used by the Reddit Search
source operator, so it belongs in operator-requirements.txt. Drop the
duplicate from requirements.txt; the operator pin (and lockfile) stay
unchanged.

Closes apache#5687
@github-actions github-actions Bot added dependencies Pull requests that update a dependency file pyamber labels Jun 13, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.75%. Comparing base (4fd395b) to head (3df0831).

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #5689   +/-   ##
=========================================
  Coverage     52.75%   52.75%           
  Complexity     2546     2546           
=========================================
  Files          1090     1090           
  Lines         42150    42150           
  Branches       4529     4529           
=========================================
  Hits          22238    22238           
  Misses        18589    18589           
  Partials       1323     1323           
Flag Coverage Δ *Carryforward flag
access-control-service 71.42% <ø> (ø) Carriedforward from 4fd395b
agent-service 34.36% <ø> (ø) Carriedforward from 4fd395b
amber 52.70% <ø> (ø) Carriedforward from 4fd395b
computing-unit-managing-service 1.65% <ø> (ø) Carriedforward from 4fd395b
config-service 56.71% <ø> (ø) Carriedforward from 4fd395b
file-service 57.06% <ø> (ø) Carriedforward from 4fd395b
frontend 47.86% <ø> (ø) Carriedforward from 4fd395b
pyamber 90.67% <ø> (-0.05%) ⬇️
python 90.74% <ø> (ø) Carriedforward from 4fd395b
workflow-compiling-service 58.69% <ø> (ø) Carriedforward from 4fd395b

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 12 worse · ⚪ 3 noise (<±5%) · 0 without baseline

CI benchmark results are noisy; treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 399 0.244 23,323/34,040/34,040 us 🔴 +12.6% / ⚪ within ±5%
🔴 bs=100 sw=10 sl=64 820 0.5 121,520/143,789/143,789 us 🔴 +17.7% / 🔴 +8.9%
🔴 bs=1000 sw=10 sl=64 930 0.568 1,073,987/1,118,744/1,118,744 us 🔴 +20.3% / 🔴 -11.3%
Baseline details

Latest main 4fd395b from 2026-06-13T21:15:43.559Z

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 399 tuples/sec 425.86 tuples/sec 411.23 tuples/sec -6.3% -3.0%
bs=10 sw=10 sl=64 MB/s 0.244 MB/s 0.26 MB/s 0.251 MB/s -6.1% -2.8%
bs=10 sw=10 sl=64 p50 23,323 us 24,156 us 23,773 us -3.4% -1.9%
bs=10 sw=10 sl=64 p95 34,040 us 30,243 us 35,177 us +12.6% -3.2%
bs=10 sw=10 sl=64 p99 34,040 us 30,243 us 35,177 us +12.6% -3.2%
bs=100 sw=10 sl=64 throughput 820 tuples/sec 957.92 tuples/sec 896.95 tuples/sec -14.4% -8.6%
bs=100 sw=10 sl=64 MB/s 0.5 MB/s 0.585 MB/s 0.547 MB/s -14.5% -8.7%
bs=100 sw=10 sl=64 p50 121,520 us 103,271 us 111,596 us +17.7% +8.9%
bs=100 sw=10 sl=64 p95 143,789 us 144,102 us 139,415 us -0.2% +3.1%
bs=100 sw=10 sl=64 p99 143,789 us 144,102 us 139,415 us -0.2% +3.1%
bs=1000 sw=10 sl=64 throughput 930 tuples/sec 1,118 tuples/sec 1,048 tuples/sec -16.8% -11.3%
bs=1000 sw=10 sl=64 MB/s 0.568 MB/s 0.682 MB/s 0.64 MB/s -16.7% -11.2%
bs=1000 sw=10 sl=64 p50 1,073,987 us 892,725 us 966,517 us +20.3% +11.1%
bs=1000 sw=10 sl=64 p95 1,118,744 us 943,636 us 1,016,694 us +18.6% +10.0%
bs=1000 sw=10 sl=64 p99 1,118,744 us 943,636 us 1,016,694 us +18.6% +10.0%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,500.96,200,128000,399,0.244,23323.40,34039.65,34039.65
1,100,10,64,20,2439.29,2000,1280000,820,0.500,121520.26,143789.08,143789.08
2,1000,10,64,20,21500.27,20000,12800000,930,0.568,1073987.13,1118743.79,1118743.79

@Yicong-Huang

Copy link
Copy Markdown
Contributor Author

cc @SarahAsad23

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file pyamber

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Drop duplicate praw entry from amber/requirements.txt

2 participants