Add opt-in TCPConnector(use_truststore=True) for OS-native trust store integration (#11705)#12702
Open
Krishnachaitanyakc wants to merge 15 commits into
Open
Add opt-in TCPConnector(use_truststore=True) for OS-native trust store integration (#11705)#12702Krishnachaitanyakc wants to merge 15 commits into
Krishnachaitanyakc wants to merge 15 commits into
Conversation
aio-libs#11705) Add a keyword-only ``use_truststore: bool = False`` parameter to ``TCPConnector``. When set to ``True``, certificate verification is delegated to the OS-native trust store via the ``truststore`` library, fixing ``CERTIFICATE_VERIFY_FAILED`` errors for users behind enterprise TLS-intercepting proxies whose root CA is installed in the macOS Keychain or Windows certificate stores but not in OpenSSL's default paths. - New dedicated optional extra: ``pip install aiohttp[truststore]`` - ``use_truststore=True`` without the dependency installed raises a friendly ``RuntimeError`` at construction time - ``use_truststore=True`` together with ``ssl=False`` raises ``ValueError`` - Explicit ``ssl=<SSLContext>`` always wins over ``use_truststore=True`` - Default behaviour is unchanged - Per-connector context instance, not a module-level singleton Also fixes a misleading sentence in ``docs/client_advanced.rst`` that claimed Python uses the system CA certificates by default (untrue on macOS, partially wrong on Windows). This is the "optional first" step from @webknjaz's structured plan in the issue; a follow-up PR can flip the default and add truststore as a hard dependency once the opt-in path has bake-in time. Tests: 12 new tests in ``tests/test_truststore.py`` covering all branches. Full ``tests/test_connector.py`` regression run: 179 passed, 7 skipped, 1 xfailed (no regression).
psf-chronographer
Bot
added
the
bot:chronographer:provided
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #12702 +/- ##
==========================================
- Coverage 98.89% 98.66% -0.24%
==========================================
Files 131 130 -1
Lines 46798 46986 +188
Branches 2426 2426
==========================================
+ Hits 46283 46357 +74
- Misses 387 488 +101
- Partials 128 141 +13
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Merging this PR will not alter performance
Comparing 
Footnotes
|
…1705) Three classes of CI failure from prior commit: 1. `mypy` (Linter job) — `Cannot find implementation or library stub for module named "truststore"`. CI does not install the optional dep, so `[mypy-truststore] ignore_missing_imports` alone is insufficient (only takes effect when mypy CAN find the module). Add per-import `# type: ignore[import-not-found,unused-ignore]` so the type-check passes whether or not the dep is present. Keep the .mypy.ini section for the case when the dep IS installed locally and somebody runs mypy with full env. 2. `flake8-requirements` (I900) — `'truststore' not listed as a requirement`. Same root cause: the optional dep is not in the default test requirements. Add `# noqa: I900` on each truststore import. Also add `truststore` (gated on Python >=3.10 cpython) to `requirements/test-common.in` so the test environment installs it and the skipif-gated tests actually run -- this also addresses the codecov patch-coverage drop from the prior commit (every truststore test was being skipped because the dep was missing). 3. `flake8-docstrings` D205/D209 — multi-line docstrings without a blank line between summary and description and with closing quotes on the body line. Collapse the affected docstrings to single-line summaries; preserve multi-line form only where there is genuine prose after the summary (and add the blank line / standalone closing quotes where preserved). Verified locally with truststore installed and uninstalled: 0 errors on `aiohttp/connector.py` and `tests/test_truststore.py` under `mypy aiohttp/connector.py tests/test_truststore.py` and `flake8 -j 1 aiohttp/connector.py tests/test_truststore.py`. Note: requirements/*.txt lock files are not regenerated in this commit because pip-compile output depends on the resolving Python version and produces off-target diffs; the maintainer can regenerate via the project's standard workflow. Refs aio-libs#11705
for more information, see https://pre-commit.ci
Dreamsorcerer
added
the
pr-unfinished
…bs#11705) Address review feedback on aio-libs#12702. Replace the opt-in `TCPConnector(use_truststore=True)` parameter with the shape requested in the issue and by maintainers: prefer `truststore` automatically when the optional library is importable, fall back to stdlib `ssl` otherwise. Model the shielded module-level import on the existing brotli pattern in `aiohttp/compression_utils.py`. - `aiohttp/connector.py`: - Shielded `try: import truststore; HAS_TRUSTSTORE = True except ImportError: HAS_TRUSTSTORE = False` at module top. - `_make_ssl_context(verified)` prefers `truststore.SSLContext` when `HAS_TRUSTSTORE` and `verified`; falls back to `ssl.create_default_context()`. `set_alpn_protocols` runs in both branches. - Remove `use_truststore` kwarg, `_use_truststore` / `_ssl_context_truststore` instance state, the `ValueError` for `ssl=False`, and the truststore branch in `_get_ssl_context`. The module-level `_SSL_CONTEXT_VERIFIED` now carries the auto-prefer behaviour. - Remove the `_import_truststore()` helper and its RuntimeError path. - `tests/test_truststore.py`: rewrite around `HAS_TRUSTSTORE` with both branches covered (installed via `pytest.importorskip` / skipif; absent via `mock.patch.object(connector_module, "HAS_TRUSTSTORE", False)`). The fallback assertion uses `type(ctx) is ssl.SSLContext` because `truststore.SSLContext` subclasses `ssl.SSLContext` and `isinstance` would pass in both branches. - `docs/client_reference.rst`: drop `use_truststore=False` from the constructor signature and the `:param`. Add a `versionchanged:: 3.14` note describing the auto-prefer. - `docs/client_advanced.rst`: rewrite the example -- `pip install aiohttp[truststore]` is all that is required; show the explicit-context escape hatch for users who want to force the stdlib trust paths. - `CHANGES/11705.feature.rst`: rewrite the fragment to describe the automatic preference rather than an opt-in flag. `truststore` remains an optional extra in `pyproject.toml`; it is not promoted to a required runtime dependency. `THREAT_MODEL.md` was inspected and does not reference the default TLS trust source, so no chunk update is included. Tests: 8 in `tests/test_truststore.py` (all pass); regression run on `tests/test_connector.py` reports `179 passed, 7 skipped, 1 xfailed`, matching the baseline.
github-actions
Bot
removed
the
pr-unfinished
Truststore's _configure_context probes well-known CA file/directory locations on every wrap_socket call. On Linux the openssl backend calls os.path.isfile / os.path.isdir / os.listdir, which blockbuster catches as blocking I/O on the event loop. Add a narrow allowance scoped to truststore/_openssl.py frames so the existing test suite passes when truststore is installed. Also add MDM and stdlib to the spelling wordlist for new prose in the truststore docs section.
for more information, see https://pre-commit.ci
New prose in the truststore docs sections introduces these tokens in bare text (outside any role); add them to docs/spelling_wordlist.txt so the docs spelling check passes.
"importable" is not in enchant's en_US dictionary; appears in the new
truststore docs prose ("whenever the library is importable").
…11705) truststore's intersphinx inventory does not expose SSLContext as a py:class target, so the :class:\`truststore.SSLContext\` references fail under -W with "py:class reference target not found". Prefix the cross- reference name with "!" to keep the inline styling without attempting to resolve the target.
bdraco
reviewed
May 31, 2026
Comment on lines
+107
to
+111
| for func, fn_names in ( | ||
| ("os.stat", ("_configure_context", "_capath_contains_certs")), | ||
| ("os.listdir", ("_capath_contains_certs",)), | ||
| ): | ||
| bb.functions[func].can_block_in("truststore/_openssl.py", fn_names) |
Member
There was a problem hiding this comment.
Looks like blockbuster is hitting in the _create_connection_transport chain and this papering over the issue. This is a blocking os.stat and an os.listdir in the call chain
Member
There was a problem hiding this comment.
Is this a limit of Python's ssl module, or something truststore could improve in some way?
bdraco
requested changes
May 31, 2026
Member
There was a problem hiding this comment.
#12702 (comment) needs to be fixed before merging especially since this automatically loads when its installed and is not truly opt-in. The import time SSLContext that is currently used is designed to avoid the blocking I/O at runtime, and this PR regresses that.
The previous iteration auto-preferred truststore whenever importable, which made the module-level _SSL_CONTEXT_VERIFIED a truststore.SSLContext and reintroduced per-handshake blocking I/O on the event loop (_configure_context runs from wrap_socket on every connection and probes os.path.isfile/isdir and os.listdir). bdraco flagged that the conftest blockbuster allowance was papering over a real regression of the existing import-time SSLContext design contract. Restore the original opt-in semantics: - _make_ssl_context grows a use_truststore keyword; the truststore branch only fires when explicitly requested. Missing dependency now raises RuntimeError at the opt-in site instead of silently falling back. - Module-level _SSL_CONTEXT_VERIFIED stays plain ssl.SSLContext, built once at import — default request path performs zero per-handshake file I/O, matching pre-PR behaviour. - TCPConnector grows use_truststore=False; when True, the truststore context is built eagerly in __init__ so dependency errors surface at construction. ssl=False + use_truststore=True raises ValueError. - _get_ssl_context prefers the per-instance truststore context when present, falls back to the module-level stdlib context otherwise. - Remove the conftest blockbuster allowance: the default path no longer exercises truststore code, and opt-in tests carry skip_blockbuster because the user has explicitly accepted that cost. - Rewrite tests/test_truststore.py around the opt-in flag with a regression guard that _SSL_CONTEXT_VERIFIED is type(ctx) is ssl.SSLContext (not just isinstance — truststore subclasses). - Update docs to document use_truststore as an opt-in parameter and call out the per-handshake I/O cost so users opt in knowingly.
for more information, see https://pre-commit.ci
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes
Closes #11705 (in part — this is the additive, opt-in step that the issue's action list anticipates as "follow pip's example and make it optional first").
What this changes
Adds a new keyword-only
use_truststore: bool = Falseparameter toTCPConnector. When set toTrue, certificate verification is delegated to the OS-native trust store via thetruststorelibrary, fixingCERTIFICATE_VERIFY_FAILEDerrors for users behind enterprise TLS-intercepting proxies whose root CA is installed in the macOS Keychain or Windows certificate stores but not in OpenSSL's default paths.pip install aiohttp[truststore](kept separate fromspeedupsbecause trust-store integration is a correctness/functionality concern, not a perf accelerator).use_truststore=Truewithout the dependency installed raises a friendlyRuntimeErroratTCPConnectorconstruction, not deep in an async handshake stack.use_truststore=Truetogether withssl=FalseraisesValueError— there is no concept of an "unverified truststore" context.ssl=<SSLContext>always wins overuse_truststore=True.ssl=,ssl_context=, or rely on defaults see zero behaviour change.truststoreis an OPTIONAL dependency. Without it installed, aiohttp works exactly as today.Why this is "step 1"
The issue body explicitly leaves the rollout strategy open:
This PR is the "optional first" step. A follow-up PR can flip the default and add truststore as a hard dependency once the opt-in path has bake-in time, and that follow-up will also expand
THREAT_MODEL.mdto cover the new default trust source. Keeping the steps separate makes the design conversation per PR small and reviewable.Documentation
docs/client_advanced.rst: fixed the misleading "By default, Python uses the system CA certificates." sentence (which is wrong on macOS, partially wrong on Windows) and added a new "Example: Use truststore for system trust stores" section explaining the corporate-proxy use case.docs/client_reference.rst: addeduse_truststoreto theTCPConnectorsignature block and a parameter description with.. versionadded:: 3.14.Tests
12 new tests in
tests/test_truststore.pycovering all branches: default off, off-no-import, on-with-truststore, on-without-truststore, ssl=False guard, explicit-context precedence, per-instance isolation, dispatch return value, fingerprint composition, unverified-path ignores flag, ALPN compatibility, and the import helper itself. Truststore-dependent tests usepytest.importorskipso the suite still runs without the optional dependency.Local test results:
12 passed, 0 failed. Fulltests/test_connector.pyregression run:179 passed, 7 skipped, 1 xfailed— no regression.Manual reproduction
End-to-end smoke test (trustme-issued self-signed cert)
TCPConnector(use_truststore=True)constructs atruststore.SSLContextinstance. A separate HTTPS round trip against a self-signedtrustme-issued cert confirms the wider SSL pipeline still works under truststore monkey-patching. Output:Checklist
CHANGES/11705.feature.rst)CONTRIBUTORS.txtCredit
Thank you @sethmlarson for the
truststorelibrary, and @webknjaz for the structured tracking issue and the open call for new volunteers.