PyGILState_Ensure does not acquires GIL

[kunoospald]

BPO	1720250
Nosy	@birkenfeld, @terryjreedy, @amauryfa, @ncoghlan, @pitrou

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-02-28.14:48:10.103>
created_at = <Date 2007-05-16.19:46:29.000>
labels = ['type-feature', 'docs']
title = 'PyGILState_Ensure does not acquires GIL'
updated_at = <Date 2011-02-28.14:48:10.102>
user = 'https://bugs.python.org/kunoospald'

bugs.python.org fields:

activity = <Date 2011-02-28.14:48:10.102>
actor = 'pitrou'
assignee = 'docs@python'
closed = True
closed_date = <Date 2011-02-28.14:48:10.103>
closer = 'pitrou'
components = ['Documentation']
creation = <Date 2007-05-16.19:46:29.000>
creator = 'kunoospald'
dependencies = []
files = []
hgrepos = []
issue_num = 1720250
keywords = []
message_count = 7.0
messages = ['32027', '57619', '109823', '109832', '109865', '129695', '129710']
nosy_count = 9.0
nosy_names = ['georg.brandl', 'terry.reedy', 'amaury.forgeotdarc', 'ncoghlan', 'pitrou', 'kunoospald', 'urBan_dK', 'glchapman21', 'docs@python']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1720250'
versions = ['Python 3.2']

[kunoospald]

The function PyGILState_Ensure doesn't acquire the GIL if current thread state is valid. In contrast to that PyGILState_Release deletes the thread state (PyThreadState_DeleteCurrent) which releases the GIL which got not acquired before (=> mutex->owned = -2).

Here is an example which locks at PyRun_SimpleString:

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

// release the GIL as PyEval_InitThreads 
// implicitly acquires the GIL
PyEval_ReleaseLock();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate); 

In that simple example the problem can be fixed by removing the call to PyEval_ReleaseLock. But that is needed for applications that call into the interpreter from multiple threads.

The only solution I could found up to that point is the following:

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

PyThreadState* tcur = PyThreadState_Get() ;

PyThreadState_Swap(NULL);
PyThreadState_Clear(tcur);
PyThreadState_Delete(tcur);

// release the GIL as PyEval_InitThreads 
// implicitly acquires the GIL
PyEval_ReleaseLock();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate);

Which seems to works fine. But I think that this behavior of PyGILState_Ensure should be either documented or fixed.

Thanks,
Kuno

[glchapman21]

In my embedding, I use the following (adapting the example above):

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

/* Swap out and return current thread state and release the GIL */
PyThreadState tstate = PyEval_SaveThread();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate); 

You don't have to free the tstate returned by PyEval_SaveThread because
it is the thread state of the main thread (as established during
Py_Initialize), and so it will be freed during Python's shut-down.

I think in general you should rarely need to call PyEval_ReleaseLock
directly; instead use PyEval_SaveThread, the Py_BEGIN_ALLOW_THREADS
macro, or PyGILState_Release (as appropriate). The documentation should
probably say as much.

[terryjreedy]

Please check whether this is still an issue in 3.1, so that there is still an issue for 3.2.

[amauryfa]

This is still the case: the documentation should mention that PyEval_ReleaseLock() is not the correct function to release "the GIL", both the interpreter lock *and* the current thread state have to be released.

[pitrou]

I'm not even sure what PyEval_AcquireLock() and PyEval_ReleaseLock() are good for. Perhaps they should be deprecated.

[ncoghlan]

Given the deprecation of PyEval_ReleaseLock in bpo-10913, should this be closed as "out of date"?

[pitrou]

Given the deprecation of PyEval_ReleaseLock in bpo-10913, should this
be closed as "out of date"?

Indeed :)