Suppose there is this system which is designed to process data.

Suppose data goes into that system via a JMS queue, where that data is processed, and depending on the results of that processing, the data is put into one of two outgoing queues.

Now suppose the following: (1) if the data isn’t correctly pulled off of the queue for whatever reason, the item is requeued, and (2) the system is massively parallel: the system doesn’t run on one computer, but on a half dozen.

Now suppose there is a bug in the system: on occasion for reasons no-one really understands, there is a flaw in the system which causes items to become ‘stuck’ in the queue: they get pulled off, then they get pushed back. This behavior should never happens–but it does.

What do you do?

Well, hypothetically you could debug the system and fix it, so items don’t get stuck in the queue.

Or you could create a “stuck in queue” report for management, to let them know that while there is a problem, it’s under control.

Really: does a report indicating the volume and scope of your problem really mean the problem is solved? Gah!