FAQs

Q: Where can I find information about upgrading to a new NiFi version?

• A: We are still need to put together a real upgrade guide; however, in the meantime, there are some notes on upgrading in the System Properties section of the Administrator Guide. These tips have to do with setting up your current NiFi in such a way that will make upgrading easier.

Q: Where can I find information about the REST API?

• A: The REST API documentation is included in the "help" documentation within the application and also on our web site here. To get to the documentation within the application, click on the "help" link in the upper-right corner of the NiFi user interface. Then, in the pane on the left-hand side, scroll down to the very bottom, where you will see a Developer section, with links to the Developer Guide and the REST API documentation.

Q: How do I select multiple items on the graph at the same time, such as if I want to select and move a group of processors?

• A: You can either select one item, and then hold down the Shift key and select other items, or you can click anywhere on the blank graph, outside what you want to select, then hold down the shift key and drag a selection box around what you want to select.

Q: ListenHTTP -

I have built most of our production flow in NiFi. Part of the data is received from thousands of distributed components via the ListenHTTP processor. I now need to add some automatic registration for these components (this assigns some ID to the component) and enable authentication using the generated ID. Is this possible to implement with the current version? The registration could happen outside (maybe using Play), but the file reception should happen in NiFi in order to initiate the flow.

• A: The ListenHTTP processor isn't so general purpose as to support custom designed authentication and authorization schemes. You could use it to restrict who can provide data through use of HTTPS and specific certificate DN patterns or to simply accept whatever data is brought in and then have a follow-on processor in the flow choose to drop data or not because it knows and trusts the source though. If you want to be able to block data as a client component tries to feed it to NiFi using a custom scheme, then you'll probably want to look at a custom processor or wire together the HandleHttpRequest and HandleHttpResponse processors. With those you can build a rather custom and powerful API/interface and have it support your own chosen authentication and authorization scheme on top of HTTP/S. With those processors if you understand HTTP well you can basically visually construct a web service. It requires the user to have a lot of knowledge to get the most of out it though.

Q: If I have FlowFiles stuck in my queue, how can I clear them out?

• A: If you are testing a flow and do not care about what happens to the test data that is stuck in a connection queue, you can reconfigure the connection and temporarily set the FlowFile Expiration to something like 1 sec so that the FlowFiles currently in the queue will expire. Then, you can reset it to 0 sec (which means never expire). Note that you must stop both processors on either side of a connection before you can edit the settings.

Please add questions about process groups here.

Q: How can I send data from one NiFi instance to another?

• A: There are generally two approaches to sending data between NiFi instances...
  - Push: The sending instance brings data to a remote process group which is pointing at the receiving NiFi, the receiving NiFi has an input port to receive the data
  - Pull: The sending instance brings data to an output port, and the receiving instance has a remote process group pointing at the output port of the sending instance
  The advantage of the first approach (push) is that the sending instance is deciding where to send its data, as opposed to data waiting at an output port for anyone to consume.

Q: I set up a two-node cluster, but after I disconnected one node, did some work in the cluster, and tried to reconnect the second node, I got the error "Failed to connect node to cluster because local flow is different than cluster flow.”  I deleted the flow.xml.gz file from the node I’m attempting to add and then restarted it, but I still had the same issue.  How do I resolve this?

• A: This problem can occur if you have templates on the second node that are not in sync with the cluster. On the second node, delete the flow.xml.gz and also delete the templates (located by default in conf/templates), then try to restart the node and see if it can connect to the cluster.

Q: I am evaluating NiFi for a large project to see if NiFi would fit as the main data collector. The project I am working on would require retrieving several hundreds of millions of files per day (hundreds of TB per day) so my first question is how to achieve distribution/clustering with NiFi, if that's possible.

• A: The NiFi Administrator Guide includes a quick explanation of our clustering capabilities and example configurations. This would be a great place to start and become familiar with NiFi clustering.

Please add questions about controller services here.

Please add questions about reporting tasks here.