The real(?) reason why the CPython GIL can't be removed

Top Page

Reply to this message
Author: Andy Sy
To: True Computer Science Mailing List, A Philippine Pythonista Hangout
Subject: The real(?) reason why the CPython GIL can't be removed
Python's inventor Guido von Rossum has always resisted
removing the GIL from Python because it is claimed that
many many fine-grained locks needed as a result saps
performance to the extent that it makes the whole
exercise pointless.

These arguments have always sounded reasonable and
crushed the hopes of those who wanted to use Python
to take advantage - in a straightforward manner (e.g.
without using non-standard libraries) - of the huge
performance benefits of using multiple threads on
multicore machines. (Remember that Ruby seems to
have a very similar Achilles heel.)

Happily, Jython and IronPython DO NOT rely on any such
global interpreter lock and as a consequence, do not
keep multithreaded apps from fully exploiting the power
of multicore/SMP devices.

The question remained though as to why jettisoning the
GIL on the CLR and JVM implementations of Python was
*such a non-event* whereas the CPython versions - even
up to the newest 3.x afaik - still require it.

I found this comment at

"The real problem is that since CPython is stuck with
using the slow reference counting instead of fast, modern,
real GCs, so removing the GIL for CPython makes it twice
as slow. Fine-grained threading is slower, but not by
that factor. Atomic manipulation of reference counts is
what killed performance of GIL-free CPython, and reference
counting already really slows down Python."

I'm not sure by exactly what mechanisms reference counting
would introduce the need for fine-grained locks whereas
having a GC wouldn't, but this seems to be a very likely
candidate explanation in light of the surrounding realities.

I was thinking back to this post

and this would also seem to explain the puzzle of the extra
core being used here:

Could the 4th core running 100% - even with only 3 worker
threads spawned - mean that it is the GC thread hard at

Web and IT Consulting
Philippine Pythonista Hangout
python@??? (#Python @
Searchable Archives: