Skip to content

perf(bench): make snap-then-simulate benchmarks faster#2775

Open
gilcu3 wants to merge 1 commit intomainfrom
2774-criterion-benchmarks-run-expensive-setup-per-sample-unnecessarily
Open

perf(bench): make snap-then-simulate benchmarks faster#2775
gilcu3 wants to merge 1 commit intomainfrom
2774-criterion-benchmarks-run-expensive-setup-per-sample-unnecessarily

Conversation

@gilcu3
Copy link
Copy Markdown
Contributor

@gilcu3 gilcu3 commented Apr 9, 2026

Closes #2774
takes 12 minutes on my machine:

❯ MAX_MALICIOUS=40 cargo bench -p threshold-signatures --bench advanced_dkg
    Finished `bench` profile [optimized] target(s) in 0.16s
     Running benches/advanced_dkg.rs (target/release/deps/advanced_dkg-433c4cf2c869f36c)
dkg/dkg_secp256k1_MAX_MALICIOUS_40_PARTICIPANTS_41
                        time:   [2.5496 ms 2.5653 ms 2.5822 ms]
Found 3 outliers among 15 measurements (20.00%)
  3 (20.00%) low mild

dkg/dkg_ed25519_MAX_MALICIOUS_40_PARTICIPANTS_41
                        time:   [2.8598 ms 2.9045 ms 2.9366 ms]
Found 2 outliers among 15 measurements (13.33%)
  2 (13.33%) low mild

dkg/dkg_bls12381_MAX_MALICIOUS_40_PARTICIPANTS_41
                        time:   [2.8873 ms 2.9659 ms 3.0107 ms]

@gilcu3 gilcu3 linked an issue Apr 9, 2026 that may be closed by this pull request
@gilcu3 gilcu3 marked this pull request as ready for review April 9, 2026 12:01
@claude
Copy link
Copy Markdown

claude bot commented Apr 9, 2026

PR title type suggestion: This PR changes only benchmark and test utility files, so the type prefix should probably be test: instead of perf:.

Suggested title: test: make snap-then-simulate benchmarks faster

@claude
Copy link
Copy Markdown

claude bot commented Apr 9, 2026

Code Review

No critical issues found. The refactoring correctly hoists expensive one-time protocol setup out of Criterion per-sample iter_batched closure, caching the Simulator and cloning it cheaply for each iteration. Key observations:

  • The ProtocolSnapshot signature change (self to &self on new and get_received_messages) is safe since get_received_messages already delegates to read_all_messages which only borrows.
  • MockCryptoRng clone preserves determinism identically to the prior approach (same seed produces same state each run).
  • derive Clone on Simulator is appropriate since it only contains a Participant (Copy) and Vec.

Approved

@gilcu3 gilcu3 changed the title perf: make snap-then-simulate benchmarks faster perf(bench): make snap-then-simulate benchmarks faster Apr 9, 2026
@claude
Copy link
Copy Markdown

claude bot commented Apr 9, 2026

PR title type suggestion: This PR changes only benchmark and test utility files, so the type prefix should probably be test: instead of perf:.

Suggested title: test: make snap-then-simulate benchmarks faster

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Criterion benchmarks run expensive setup per sample unnecessarily

2 participants