Unable to get local issuer certificate when using twine

[wqh17101]

Hi there, i am using

export CURL_CA_BUNDLE=""
twine upload xxx

to upload packages.
What is strange is that i can use this cmd successfully with twine==4.0.1 requests==2.27.1 certifi==2021.10.8 or 2022.6.15。
Failed with twine==4.0.1 requests==2.28.1 certifi==2021.10.8 or 2022.6.15。
Also

export CURL_CA_BUNDLE=""
export REQUESTS_CA_BUNDLE=""
twine upload xxx

failed with requests==2.28.1

Expected Result

Using twine upload xxx successfully.

Actual Result

Traceback (most recent call last):
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/abc/python/python-3.9.2/lib/python3.9/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/abc/python/python-3.9.2/lib/python3.9/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/abc/python/python-3.9.2/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connectionpool.py", line 815, in urlopen
    return self.urlopen(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connectionpool.py", line 815, in urlopen
    return self.urlopen(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connectionpool.py", line 815, in urlopen
    return self.urlopen(
  [Previous line repeated 7 more times]
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='pypi.cloudartifact.dgg.dragon.tools.huawei.com', port=443): Max retries exceeded with url: /artifactory/api/pypi/pypi-oss/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/abc/python/python-3.9.2/bin/twine", line 8, in <module>
    sys.exit(main())
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/twine/__main__.py", line 33, in main
    error = cli.dispatch(sys.argv[1:])
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/twine/cli.py", line 123, in dispatch
    return main(args.args)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/twine/commands/upload.py", line 198, in main
    return upload(upload_settings, parsed_args.dists)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/twine/commands/upload.py", line 142, in upload
    resp = repository.upload(package)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/twine/repository.py", line 186, in upload
    resp = self._upload(package)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/twine/repository.py", line 172, in _upload
    resp = self.session.post(
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/abc/python/python-3.9.2/lib/python3.9/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='pypi.cloudartifact.dgg.dragon.tools.huawei.com', port=443): Max retries exceeded with url: /artifactory/api/pypi/pypi-oss/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))

Reproduction Steps

export CURL_CA_BUNDLE=""
twine upload xxxx

System Information

$ python -m requests.help

2.27.1

{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.2"
  },
  "platform": {
    "release": "4.19.36-vhulk1907.1.0.h619.eulerosv2r8.aarch64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

2.28.1

{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.2"
  },
  "platform": {
    "release": "4.19.36-vhulk1907.1.0.h619.eulerosv2r8.aarch64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

[wqh17101]

After i debug from source code, i found that this a bug.

requests/requests/sessions.py

Lines 765 to 770 in 177dd90

	if verify is True or verify is None:
	verify = (
	os.environ.get("REQUESTS_CA_BUNDLE")
	or os.environ.get("CURL_CA_BUNDLE")
	or verify
	)

from these codes, if i set REQUESTS_CA_BUNDLE="" CURL_CA_BUNDLE="" verfiy=None and verify will equal to None

>>> ("" or "" or None) is None
True

After that at

requests/requests/sessions.py

Line 775 in 177dd90

verify = merge_setting(verify, self.verify)

requests/requests/sessions.py

Lines 61 to 86 in 177dd90

	def merge_setting(request_setting, session_setting, dict_class=OrderedDict):
	"""Determines appropriate setting for a given request, taking into account
	the explicit setting on that request, and the setting in the session. If a
	setting is a dictionary, they will be merged together using `dict_class`
	"""
	if session_setting is None:
	return request_setting
	if request_setting is None:
	return session_setting
	# Bypass if not a dictionary (e.g. verify)
	if not (
	isinstance(session_setting, Mapping) and isinstance(request_setting, Mapping)
	):
	return request_setting
	merged_setting = dict_class(to_key_val_list(session_setting))
	merged_setting.update(to_key_val_list(request_setting))
	# Remove keys that are set to None. Extract keys first to avoid altering
	# the dictionary during iteration.
	none_keys = [k for (k, v) in merged_setting.items() if v is None]
	for key in none_keys:
	del merged_setting[key]

when request_setting is None, it will return session_setting which is self.verify that is True by default.

So that is why REQUESTS_CA_BUNDLE="" CURL_CA_BUNDLE="" verfiy=None will not work.

[sigmavirus24]

Related to #6071 which was documented in our changelog

[wqh17101]

@sigmavirus24 That is too long to read. Maybe you can tell me the conclusion.
Two question:
1.Is this a bug to fix in the future?
2.If this is not a bug,how to skip verify by cmd line.

[sigmavirus24]

1.Is this a bug to fix in the future?

No.

2.If this is not a bug,how to skip verify by cmd line.

That's not a feature that's planned, as indicated by that thread you refuse to read. It also has the myriad reasons which I will not summarize for you. I also don't think this is a valid bug report, so I'm closing it