Setting REQUESTS_CA_BUNDLE to directory causes exception #2731

Describe the bug

The requests library supports setting REQUESTS_CA_BUNDLE to either a single certificate file, or to a directory of files. It appears that botocore assumes this value will only be a file and raises an exception if it is set to a directory.

Requests Documentation regarding REQUESTS_CA_BUNDLE: https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification

Expected Behavior

botocore to function properly whether REQUESTS_CA_BUNDLE is set to a file or a directory path.

Current Behavior

botocore raises an exception when REQUESTS_CA_BUNDLE is set to a directory path.

Reproduction Steps

Successful execution with REQUESTS_CA_BUNDLE set to a single file:

REQUESTS_CA_BUNDLE=/usr/local/lib/python3.10/site-packages/certifi/cacert.pem aws s3 ls
An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

Please disregard the Access Denied error, that is expected.

Unsuccessful execution with REQUESTS_CA_BUNDLE set to a directory:

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ aws s3 ls
SSL validation failed for https://s3.us-east-2.amazonaws.com/ [Errno 21] Is a directory

Possible Solution

One working solution I've found with some local experimentation is to modify _setup_ssl_cert to check whether a given path is a directory and set the connection parameters accordingly.

Before:

    def _setup_ssl_cert(self, conn, url, verify):
        if url.lower().startswith('https') and verify:
            conn.cert_reqs = 'CERT_REQUIRED'
            conn.ca_certs = get_cert_path(verify)
        else:
            conn.cert_reqs = 'CERT_NONE'
            conn.ca_certs = None

After:

    def _setup_ssl_cert(self, conn, url, verify):
        if url.lower().startswith('https') and verify:
            conn.cert_reqs = 'CERT_REQUIRED'
            cert_path = get_cert_path(verify)
            if os.path.isdir(cert_path):
                conn.ca_cert_dir = cert_path
            else:
                conn.ca_certs = cert_path
        else:
            conn.cert_reqs = 'CERT_NONE'
            conn.ca_certs = None

With the above code in place, I can now see successful command executions when REQUESTS_CA_BUNDLE is set to either a file or a directory path.

REQUESTS_CA_BUNDLE=/usr/local/lib/python3.10/site-packages/certifi/cacert.pem aws s3 ls
An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ aws s3 ls
An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied

Again, please disregard the Access Denied error, that is expected.

Additional Information/Context

No response

SDK version used

botocore==1.27.43

Environment details (OS name and version, etc.)

Debian 11, Python 3.10.5