Rails threading nightmare
bibwild.wordpress.com
bibwild.wordpress.com
This works, is fairly simple to implement, and kicks concurrency to database programmers, who actually understand it.
The only reason I hesitate to give this advice is that substantially all of my downtime can be traced to worker processes either failing silently or going wild and munching my swap until the server died. Productizing the use of them is a little tricky.
Concrete recommendation: I like DelayedJob, Thomas likes the Redis queue.
We had exactly the problem this post describes with Playbook, even without using threads. We suffered random MRI crashes for weeks and weeks. In particular, the SWIG bindings for Subversion (two great tastes that make me want to throw up in my mouth) would randomly corrupt memory, but only when hosted in a process servicing Rails requests.
We solve this problem --- remarkably easily --- by creating a shim class in front of our SVN code, hosting the "real" SVN code in a sidecar process, and remoting the requests. In no other mainstream language is it as easy to remote requests; you can literally just Marshal.save({method => args}) and chuck that over a socket. It's even easy to pass exceptions back and forth.
If you have a task-based architecture, there's already a Rails best practice solution: use DJ or Resque to factor tasks out into worker classes that run off job queues. Both DJ and Resque are extremely easy to set up. There's no reason to overthink this.
That could either kill your threads or prevent the webserver from correctly shutting down the process or handling another request on the same process.
I would hand the background jobs off to some daemon process or write instructions describing the job into some queue and then have a cron job process them.
I would never expect to be able to keep anything running after request processing has finished.
class Future
def initialize(postprocessor = nil, &finalize)
@postprocessor = postprocessor
@future = Thread.new do
begin
finalize.call
rescue => ex
warn "Exception in background thread: #{ex}"
end
end
end
def method_missing(method, *args, &block)
data.send(method, *args, &block)
end
def to_s
data.to_s
end
private
def data
@data ||= @postprocessor ? @postprocessor.call(@future.value) : @future.value
end
end
However this was causing random 5 second delays in my Rails app -- even if I never called "data" (and hence never joined). I added the postprocessor so that I could pass in a lambda that does all of the ActiveRecord stuff back in the calling thread, which alleviated the problem.But in contrast to what bensummers said -- this is actually a Rails problem, not a Ruby problem. Ruby's threads suck, but it seems to be ActiveRecord's locking that's at the root of the problem.
I know nothing about Ruby, so perhaps I am misunderstanding, but this would explain what he is seeing and is consistent with what he describes.
I would launch a stand-alone ruby process for the communication and connect to it via Drb to get results. Note that this will not work if you have greater than about 40-100 things trying to connect to the object, but it is simple and will work pretty well at small scales-it is less hairy than threads.
Alternately, you could put rabbitmq or a messaging queue in the middle, and look at EventMachine for the communicator process. That will scale nicely.
Not that threading is perfect. But really, if you're doing anything intensive, you should be switching to background job + polling, anyway. You don't want to starve your mongrels/unicorn workers!
This isn't to say that your background job wouldn't have concurrency... but it doesn't need the whole Rails environment.
Ruby threading has changed a lot since 2009. Still not perfect, but Rubinius is supposed to be removing the GIL...
Switching to JRuby would be an easy solution to these woes. Most gems are compatible these days, and there's even experimental support for C extensions as a temporary solution during migration.