How do Enumerators work in Ruby 1.9.1?

Actually in your e = Bunk.new.each the else clause is not executed initially. Instead the 'if !block_given' clause executes and returns an enumerator object. The enumerator object does keep a fiber object internally. (At least that is what it looks like in enumerator.c)

When you call e.each it is calling a method on an enumerator which uses a fiber internally to keep track of its execution context. This method calls the Bunk.each method using the fibers execution context. The Bunk.each call here does execut the else clause and yields up the value.

I do not know how yield is implemented or how a fiber tracks the execution context. I haven't looked at that code. Almost all of the enumerator and fiber magic is implemented in C.

Are you really asking how fibers and yield are implemented? What level of detail are you looking for?

If I am off base please correct me.

As the other posters noted, I believe it creates its own private "fiber" [in 1.9]. In 1.8.7 (or 1.8.6 if you use the backports gem) somehow or other it does the same thing (perhaps because all threads in 1.8 are the equivalent of fibers, it just uses them?)

Thus in 1.9 and 1.8.x, if you chain several of them together a.each_line.map.each_with_index { }

It actually flows through that whole chain with each line, kind of like a pipe on the command line

http://pragdave.blogs.pragprog.com/pragdave/2007/12/pipelines-using.html

HTH.

Here's a plain ruby enumerator that uses Fibers and should pretty much behave like the original:

class MyEnumerator
  include Enumerable

  def initialize(obj, iterator_method)
    @f = Fiber.new do
      obj.send(iterator_method) do |*args|
        Fiber.yield(*args)
      end
      raise StopIteration
    end
  end

  def next
    @f.resume
  end

  def each
    loop do
      yield self.next
    end
  rescue StopIteration
    self
  end
end

And in case anyone is feeling uneasy about using exceptions for control flow: The real Enumerator raises StopIteration at the end, too, so I just emulated the original behaviour.

Usage:

>> enum = MyEnumerator.new([1,2,3,4], :each_with_index)
=> #<MyEnumerator:0x9d184f0 @f=#<Fiber:0x9d184dc>
>> enum.next
=> [1, 0]
>> enum.next
=> [2, 1]
>> enum.to_a
=> [[3, 2], [4, 3]]