symbol not found on macOS 11

[BerryKeers]

Describe the bug
I've tried installing keyring with pip3 globally and also with pipenv, both times whenever I try to use this module i'm getting the same error:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keyring/backends/macOS/__init__.py", line 38, in set_password
    api.set_generic_password(self.keychain, service, username, password)
NameError: name 'api' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/Berry/Projects/pipenv/IEX/creds.py", line 20, in <module>
    save_cred()
  File "/Users/Berry/Projects/pipenv/IEX/creds.py", line 9, in save_cred
    keyring.set_password(service_name, 'username', username)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keyring/core.py", line 60, in set_password
    get_keyring().set_password(service_name, username, password)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keyring/backends/macOS/__init__.py", line 39, in set_password
    except api.KeychainDenied as e:
NameError: name 'api' is not defined

To Reproduce
My little test script is:

import getpass
import keyring

def save_cred():
    service_name = 'Python_IEX'
    username = input('What\'s your IEX username?\n')
    password = getpass.getpass(prompt='Password: ', stream=None)

    keyring.set_password(service_name, 'username', username)
    keyring.set_password(service_name, username, password)

    print(f'Credentials for user {username} saved successfully.')

def get_cred():
    username = keyring.get_password(service_name, 'username')
    password = keyring.get_password(service_name, 'password')

    return username, password

save_cred()

Expected behavior
A clear and concise description of what you expected to happen.

Environment

• OS: [e.g. macOS]
  MacOS Big Sur 11.5.2

$ pip list | grep keyring
keyring                           23.1.0
$ keyring --list-backends
dlsym(RTLD_DEFAULT, SecItemAdd): symbol not found
keyring.backends.chainer.ChainerBackend (priority: -1)
keyring.backends.macOS.Keyring (priority: 5)
keyring.backends.fail.Keyring (priority: 0)

[gtmanfred]

I am also experiencing this symbol not found bug.

[ShawnROGrady]

I noticed this immediately after updating to big sur, appears to be related to https://stackoverflow.com/questions/62587131/macos-big-sur-python-ctypes-find-library-does-not-find-libraries-ssl-corefou

[mitya57]

Can you paste an error that this command gives you: python3 -c 'import keyring.backends.macOS.api'?

Cc @jaraco because you are an actual macOS user unlike me.

[goldstar611]

Can you paste an error that this command gives you: python3 -c 'import keyring.backends.macOS.api'?

Cc @jaraco because you are an actual macOS user unlike me.

I just pip3 installed keyring and also receive this error.

user@user-MBP:~$ python3 -c 'import keyring.backends.macOS.api'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/keyring/backends/macOS/api.py", line 45, in <module>
    SecItemAdd = _sec.SecItemAdd
  File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, SecItemAdd): symbol not found

[goldstar611]

Upgrading to Python 3.9.7 resolved the issue for me. python3 -c 'import keyring.backends.macOS.api' returns no errors now

[lukeallpress]

Same issue

user@mbp: ~ % python3 -c 'import keyring.backends.macOS.api'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.8/site-packages/keyring/backends/macOS/api.py", line 26, in <module>
    CFDictionaryCreate = _found.CFDictionaryCreate
  File "/opt/anaconda3/lib/python3.8/ctypes/__init__.py", line 386, in __getattr__
    func = self.__getitem__(name)
  File "/opt/anaconda3/lib/python3.8/ctypes/__init__.py", line 391, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, CFDictionaryCreate): symbol not found

I'm not interested in upgrading my python beyond what Anaconda has released.

[goldstar611]

I'm not interested in upgrading my python beyond what Anaconda has released.

I think it's out of the maintainers hands since this is an underlying issue with python's ffi library when used with OSX Big Sur. According to https://bugs.python.org/issue41100 python 3.8.10 will get the fix. According to https://docs.anaconda.com/anaconda/packages/py3.8_osx-64/ ships with python 3.8.8 currently.

[lukeallpress]

Thanks for the clarity and doing my research for me! I hadn't put the pieces together.

[jaraco]

I can confirm the error is shown when running the tests on Python 3.8.5:

__________________________________________________________ ERROR collecting keyring/backends/macOS/api.py ___________________________________________________________
keyring/backends/macOS/api.py:45: in <module>
    SecItemAdd = _sec.SecItemAdd
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ctypes/__init__.py:386: in __getattr__
    func = self.__getitem__(name)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ctypes/__init__.py:391: in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
E   AttributeError: dlsym(RTLD_DEFAULT, SecItemAdd): symbol not found

And also confirm that updating to Python 3.8.10 (universal binary) fixes the issue. Since 3.9.5 was released on the same date, I expect the fix to require that version as well.

Checking the changelog for 3.9.5, I don't see anything relevant. My guess is that the universal binary is required for Python 3.9.1 and 3.8.7 or later.

I don't believe this library can address the root cause, but it may be useful for this library to disable the backend when the API is unavailable (opening up the possibility of another macOS backend that bypasses the API).

[jaraco]

I filed #529 to track the possible enhancement to not fail with the error reported above.

[goldstar611]

Checking the changelog for 3.9.5, I don't see anything relevant.

It's there, but in the full HTML view as bpo-41100, https://docs.python.org/release/3.9.5/whatsnew/changelog.html#id33

[jaraco]

Right, so it's not a specific Python version that matters but you must have a universal binary, which was introduced in Python 3.9.1 and 3.8.7, but the error will still occur on the latest releases of Python 3.8 and 3.9 if not using the universal binary, as observed above with reports of the failure on Python 3.9.2 and 3.8.8.

[daweimau]

@jaraco is this an example of a universal binary installer that you'd expect to fix this issue?

[jaraco]

Yes. The “universal2” in the name is the clue. These installers were introduced as part of the ARM support, and they seem to have the necessary symbols. For Python 3.10 and maybe 3.9, it may be your only option.