Interactive Queue Management

Goals

• Provide the ability for a dataflow manager to observe and interact with details of each connection/queue at runtime.

Background and strategic fit

Today in the NiFi ui an operator can view the size of a queue but has no ability to see or interact with the data in the queue. Data in the queue is 'live' in the flow and for operators that want to see what is flowing in real-time this would be very valuable.

It would be helpful to be able to pull items from the queue (download), place items directly into the queue (upload), observe the content and attributes of items in the queue as it is live, and to observe and alter the ordering of items within the queue. 

The ability to remove items from the queue is also desirable to deal with potential ‘clogging’ data, which exposes some bug in a processor.   It should also be made much easier for an operator to simply wipe the entire queue as today they have to stop the producer/consumers to the queue and then set expiration to let items age off then restore settings.  Being able to do so would further advance the goals and effect of highly interactive command and control of NiFi.

Assumptions

Requirements

User interaction and design

Questions

Below is a list of questions to be addressed as a result of this requirements document:

Not Doing