The real(?) reason why the CPython GIL can't be removed

Top Page

Reply to this message
Author: Andy Sy
Date:  
To: True Computer Science Mailing List, A Philippine Pythonista Hangout
Subject: The real(?) reason why the CPython GIL can't be removed
Python's inventor Guido von Rossum has always resisted
removing the GIL from Python because it is claimed that
many many fine-grained locks needed as a result saps
performance to the extent that it makes the whole
exercise pointless.

http://www.artima.com/forums/flat.jsp?forum=106&thread=214235

These arguments have always sounded reasonable and
crushed the hopes of those who wanted to use Python
to take advantage - in a straightforward manner (e.g.
without using non-standard libraries) - of the huge
performance benefits of using multiple threads on
multicore machines. (Remember that Ruby seems to
have a very similar Achilles heel.)

Happily, Jython and IronPython DO NOT rely on any such
global interpreter lock and as a consequence, do not
keep multithreaded apps from fully exploiting the power
of multicore/SMP devices.

The question remained though as to why jettisoning the
GIL on the CLR and JVM implementations of Python was
*such a non-event* whereas the CPython versions - even
up to the newest 3.x afaik - still require it.

I found this comment at http://www.grouplens.org/node/244

"The real problem is that since CPython is stuck with
using the slow reference counting instead of fast, modern,
real GCs, so removing the GIL for CPython makes it twice
as slow. Fine-grained threading is slower, but not by
that factor. Atomic manipulation of reference counts is
what killed performance of GIL-free CPython, and reference
counting already really slows down Python."

I'm not sure by exactly what mechanisms reference counting
would introduce the need for fine-grained locks whereas
having a GC wouldn't, but this seems to be a very likely
candidate explanation in light of the surrounding realities.

I was thinking back to this post

http://archives.free.net.ph/message/20070908.202417.d57f97b5.en.html

and this would also seem to explain the puzzle of the extra
core being used here:

http://archives.free.net.ph/message/20100411.013826.ee6c3a0c.en.html

Could the 4th core running 100% - even with only 3 worker
threads spawned - mean that it is the GC thread hard at
work?


========================
http://www.neotitans.com
Web and IT Consulting
____________________________________________________
Philippine Pythonista Hangout
python@??? (#Python @ irc.free.net.ph)
http://lists.free.net.ph/mailman/listinfo/python
Searchable Archives: http://archives.free.net.ph