crypto: swap reqwest/hyper-tls to rustls in 3 crates#35845
Draft
jasonhernandez wants to merge 21 commits intoMaterializeInc:mainfrom
Draft
crypto: swap reqwest/hyper-tls to rustls in 3 crates#35845jasonhernandez wants to merge 21 commits intoMaterializeInc:mainfrom
jasonhernandez wants to merge 21 commits intoMaterializeInc:mainfrom
Conversation
Add bin/lint-openssl to detect all openssl dependencies, feature flags, and source imports across the workspace. This is the first step toward migrating from openssl to rustls—it serves as a tracking tool for migration progress and can later be promoted to a CI gate. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add migration plan (doc/developer/openssl-to-rustls-migration.md) with tiered breakdown of all 28 affected crates, dependency graph, replacement crate mapping, and links to Linear issues (SEC-176 through SEC-200). Include raw linter output snapshots (.txt and .json) as baseline for tracking progress. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Replace all non-FIPS crypto crate recommendations (ring, sha2, hmac, pbkdf2, subtle, rsa, ed25519-dalek, aes+cbc) with aws-lc-rs equivalents. Add FIPS 140-3 strategy section, workspace fips feature flag (SEC-201), and updated replacement crate mapping table. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add [[bans.deny]] entries for crypto crates that are not FIPS 140-3 validated: sha2, hmac, subtle, ring, pbkdf2, ed25519-dalek, aes, cbc, rsa. All new crypto code must use aws-lc-rs instead. Existing workspace and third-party usage is allowed via wrappers, with TODO comments to remove them as each crate is migrated to aws-lc-rs. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add bin/lint-fips-containers to scan Dockerfiles for FIPS 140-3 compliance gaps: non-FIPS base images, crypto-relevant package installations, and non-FIPS algorithms in cert generation scripts. Distinguishes production images (must comply) from test/dev (informational). Supports --strict and --json flags. Current results: 8 production findings across 4 base images. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Covers all three compliance layers: Rust binaries (137 openssl findings across 28 crates + sha2/hmac/subtle), container images (8 production findings across 4 base images), and Kubernetes/Helm deployment (Ed25519, image validation, external services, FIPS toggle). Includes full issue inventory (SEC-176 through SEC-213), remediation strategy, recommended execution order, and FIPS validation caveat. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Remove the rustls ban from deny.toml, unblocking all openssl-to-rustls migration work. Add `aws-lc-rs` as an optional dependency in mz-ore with two feature flags: - `crypto`: enables aws-lc-rs in standard mode - `fips`: enables aws-lc-rs with FIPS 140-3 validated module mz-ore is the natural distribution channel since every crate in the workspace depends on it. Downstream crates enable `mz-ore/crypto` (or `mz-ore/fips` for FIPS builds) to get the validated backend. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Contributor
|
Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone. PR title guidelines
Pre-merge checklist
|
2 tasks
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
jasonhernandez
force-pushed
the
jason/tier2-dep-swaps
branch
from
6e61ff4 to
747f75a
Compare
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
jasonhernandez
force-pushed
the
jason/tier2-dep-swaps
branch
from
747f75a to
b919716
Compare
The `fips` feature on mz-ore enables `aws-lc-rs/fips`, which pulls in `aws-lc-fips-sys`. That crate builds BoringSSL's FIPS module via cmake, requiring Go for integrity verification. Since cargo-test runs with `--all-features`, Go must be available in the CI builder. Fixes SEC-232. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
jasonhernandez
force-pushed
the
jason/tier2-dep-swaps
branch
2 times, most recently
from
0fdf066 to
22e1d8e
Compare
uuid 1.23.0 changed error message format which breaks the fmt_ids test in mz-persist-types. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
jasonhernandez
force-pushed
the
jason/tier2-dep-swaps
branch
from
22e1d8e to
c9755c0
Compare
Pin three deps that were inadvertently bumped during Cargo.lock regeneration: - os_info 3.11.0: avoids objc2 0.6.x which causes E0275 on macOS - chrono-tz 0.8.1: avoids Egypt timezone data change that breaks test_pg_timezone_abbrevs - serde_path_to_error 0.1.8: avoids error message format change that breaks test_mcp_observatory Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Switch TLS backend from openssl/native-tls to rustls: - mz-cloud-resources: kube openssl-tls → rustls-tls - mz-npm: reqwest native-tls-vendored → rustls-tls-webpki-roots-no-provider - mz-testdrive: reqwest native-tls-vendored → rustls-tls-webpki-roots-no-provider Uses the -no-provider variant for reqwest to avoid pulling in ring, allowing aws-lc-rs to serve as the crypto provider instead. Deferred: tiberius (SEC-223, fork needs rustls fix), segment and duckdb (no rustls feature available), storage-types (has direct native-tls dep). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The client-legacy feature was previously activated transitively. After Cargo.lock regeneration, the transitive activation stopped and `hyper_openssl::client` became configured out. Enable it explicitly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
jasonhernandez
force-pushed
the
jason/tier2-dep-swaps
branch
from
86189ae to
3506539
Compare
The webpki-roots 1.0.6 crate uses the CDLA-Permissive-2.0 license, which is already allowed in deny.toml but was missing from about.toml (the cargo-about config that must be manually kept in sync). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- mz-aws-util: remove custom hyper-tls HTTP client override; the AWS SDK already uses rustls by default, so the native-TLS override was unnecessary - mz (CLI): reqwest default-tls → rustls-tls-webpki-roots-no-provider - mz-persist: reqwest default-tls → rustls-tls-webpki-roots-no-provider Deferred: mz-dyncfg-launchdarkly (LD SDK takes hyper_tls::HttpsConnector directly — needs upstream/fork change), mz-persist openssl-sys removal (has openssl_sys::init() hack that needs investigation), mz CLI openssl-probe removal (needs source changes for cert discovery). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Remove references to native-TLS policy override and hyper-tls dep in generated docs. The AWS SDK's default rustls client is now used directly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The client-legacy feature was previously activated transitively through hyper-tls in mz-ore. After replacing hyper-tls with hyper-rustls, the transitive activation stopped and `hyper_openssl::client` became configured out. Enable it explicitly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The PR removed the custom hyper-tls HTTP client from aws-util but didn't enable the `default-https-client` feature on aws-config. With default-features = false, no HTTP client was bundled, causing environmentd to crash on startup. Also pin os_info to 3.11.0 to avoid pulling in objc2 0.6.x which causes E0275 overflow on macOS clippy due to its blanket IntoIterator impl on Retained<T>. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
jasonhernandez
force-pushed
the
jason/tier2-dep-swaps
branch
from
3506539 to
a172e9c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
reqwestfromdefault-tlstorustls-tlsin aws-util, mz CLI, and persistDepends on #35844.
Part 4/7 of the FIPS 140-3 compliance mode migration.
Test plan
cargo deny check licenses bans sourcespasses🤖 Generated with Claude Code