examining items in a python Queue

It is NOT safe to simply access the underlying queue.

The safe way to do it is to extend the Queue class. If you return the underlying dequeue object, you will NOT be getting a copy, you get the live object.

The result of this is that it can change while you are iterating it - which will result in an exception if another thread inserts into the queue during your iteration.

Knowing that python uses the GIL, you can safely use list(q.queue), because list() will never cause a context switch.

It's better to use the same lock the get() function uses, and not make assumptions about the GIL:

import queue
    
class SnapshotQueue(queue.Queue):
    def snapshot(self):
        with self.mutex:
            return list(self.queue)

That class can be used safely instead of a regular queue, and it will return a snapshot of the queue state... within a mutex and without causing issues in the underlying queue operation.


The Queue module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads.

As you can see, the Queue module was created specifically for use with threads, providing only FIFO, LIFO and priority queues, none of which provide this functionality. However by examining the source code of the Queue module, you can see that it simply uses a collections.deque (double ended queue) which can easily accomplish your task. You may index the first item ([0]) and .popleft() in constant time.


queue_object.queue will return copy of your queue in a deque object which you can then use the slices of. It is of course, not syncronized with the original queue, but will allow you to peek at the queue at the time of the copy.

There's a good rationalization for why you wouldn't want to do this explained in detail in this thread comp.lang.python - Queue peek?. But if you're just trying to understand how Queue works, this is one simple way.

import Queue
q = Queue.Queue()
q.push(1)
q.put('foo')
q.put('bar')
d = q.queue
print(d)
deque(['foo', 'bar'])
print(d[0])
'foo'

Tags:

Python

Queue