ruby working on array elements in groups of four

If I read you right, you want to have no more than 4 threads processing at a time.

Sounds to me like you should launch only 4 threads, and have them all read from a shared Queue (part of the standard thread lib) to process the elements.

You can have the threads end when the queue is empty.

Slicing the array into 4 equal arrays, and having each thread process 1/4 of the elements assumes that each element processes in the same time. If some take longer than others, some of your threads will finish early.

Using a queue, no thread stops until the shared queue is empty, so it is I think a more efficient solution.

Here is a working program based on your code to demonstrate:

require 'thread'

elements = [1,2,3,4,5,6,7,8,9,10]

def process(element)
    puts "working on #{element}"
    sleep rand * 10
end

queue = Queue.new
elements.each{|e| queue << e }

threads = []
4.times do
    threads << Thread.new do
      while (e = queue.pop(true) rescue nil)
        process(e)
      end
    end
end

threads.each {|t| t.join }

You can enumerate in groups of 4 for an array:

>> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12].each_slice(4) {|a| p a}
[1, 2, 3, 4]
[5, 6, 7, 8]
[9, 10, 11, 12]

So you can try something like

elements.each_slice(4) do | batch |
    batch.each do | element |
        threads.push(Thread.new{process(element)}}

    end
    (do stuff to check to see if the threads are done, otherwise wait )
end

Its may not be what you need, though - I have been up since 3 AM and I only had a couple of hours sleep. :/

In rails(not Ruby) a more readble form can be used in_groups_of

arr= [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
arr.in_groups_of(4, false) {|a| p a}

result:

[1, 2, 3, 4]
[5, 6, 7, 8]
[9, 10, 11]

The last row has only 3 elements as we have specified false as second argument in in_group_of. If you want nil or any other value you can replace false with that value.