Thursday, August 21, 2014

Multi-Core, Languages and the Dreaded GIL

For a very long time[1], I have always thought that the only way to deal with the coming multi-core explosion is to move to a language that deals with concurrency as a core feature in the language. This is hardly a novel idea. It has always been obvious that threaded code in standard languages is next to impossible to write correctly. This implies that the only way to deal with threads is to hide them in an abstraction in which you can devote enough resources to ensure a reasonably robust implementation. Writing thread safe code is extremely difficult and requires a huge payback to justify the effort. It's exactly the same reasoning behind allowing the kernel to do memory and process management, rather than having every application write it's own OS.

In the face of this I have made a serious effort to learn Erlang, Go and dip my toe in functional languages in an attempt to be ready for adapting to this multi-core world. As much as I love Ruby,
it was obvious that it simply would not be a reasonable solution for a machine with 32 cores and above. It and many of the other "scripting" languages, depend on having a lock on the dreaded GIL[2]. This is generally not a problem for code that runs websites due to the fact that they are largely I/O bound[3]. However, for any other workload that requires CPU these languages would be relegated to the "fancy shell script" problem domain. You simply wouldn't write anything serious in these languages, since they can't effectively use 80-90% of the available resources on a high end server.

Recently, I have begun to rethink this assumption.  The reality is that most 32 core machines are running VM's in one form or another. Docker[4], in particular, provides a highly efficient means of simulating a single core OS for a single core process. A common theme of the "multi-core" ready languages is to avoid the problems of threading by switching the model from shared memory to very fast messaging. If it is possible to create an interprocess message system between Docker images that can compete with speed of Go channels or Erlang messaging, then it should be possible for languages like Ruby to be a reasonable alternative in this brave new world.

There was an attempt to allow this kind of distributed processing in Ruby, DRb. It never really gained wide usage primarily due to the latency and AAA[5] issues. However, if there was an API that allowed communication between Docker instances that could be both fast and trusted, it would be possible to revive Drb and similar "remote object" implementations. I think this is the only workable way forward for the "interpreted" languages. Essentially you have to convert the message sending primitive to sending a message to remote[6] objects. This isn't as hard as it looks at first blush, Docker images are simply processes running in the same kernel on the same machine. They just have different ideas about their various interfaces to the rest of the world. It should be possible to "short-circuit" these interfaces to allow them to communicate at something close to the speeds available to more concurrency aware languages.

[ After a couple days of thinking about what this really means.]

This idea looks more like "sprinkle current fairy dust" on a hard problem, than an actual solution. I think what I was trying to wrap my brain around was the idea of implementing something like the old sun RPC interface via standard ports as a way to allow you to use Docker to avoid all the overhead of
doing message passing between ruby processes via pipes, etc...

I think there is something there, but it's far from fleshed out enough. Maybe just creating a lightweight micro services API and using Docker to allow those API's to talk on the same machine
might be enough. I am well aware of the first rule of Distributed Objects.

[1]- Since about 1997 or so (i.e. when I first attempted to write pthreaded code ).

[2]- Global Interpreter Lock

[3]- i.e. Turning database entries into web pages and web forms into database entries.

[4] -

[5]- Authentication, Authorization  and Accounting

[6]- If they are in the same memory accessed by the same cpu, they really aren't "remote"