Please post questions/answers under the following categories. If you don't see an appropriate category for your addition, please add one.
General Application/Framework
Expand |
---|
Q: Where can I find information about upgrading to a new NiFi version?
Q: Where can I find information about the REST API?
Q: What is the base endpoint for the NiFi REST API?
Q: How do I select multiple items on the graph at the same time, such as if I want to select and move a group of processors?
Q: How do I set up NiFi to run behind a proxy?
Q: Am I correct in assuming that I can transit large volumes of data through NiFi flows in and out of Hadoop?
Q: What happens to my data if there is a power loss and the system goes down?
Q: How do I enable debug logging for a specific processor, rather than system-wide?
Q: I want to know how I can package and deploy the same dataflow from a development environment to a testing environment. Do I need to recreate the entire dataflow again in the different environment?
Q: At what point is a piece of data considered under NiFi's control?
Q: How do I bend connections so that I can create nicer-looking dataflows?
Q: If no prioritizers are set in a processor, what prioritization scheme is used?
|
Processors
Expand |
---|
Q: ListenHTTP - I have built most of our production flow in NiFi. Part of the data is received from thousands of distributed components via the ListenHTTP processor. I now need to add some automatic registration for these components (this assigns some ID to the component) and enable authentication using the generated ID. Is this possible to implement with the current version? The registration could happen outside (maybe using Play), but the file reception should happen in NiFi in order to initiate the flow.
Q: GetSFTP - I'm setting up a GetSFTP processor to run every morning to download files expected in a regular location that is named based on the previous day's date, e.g. /logs/2015-09-08. I'm trying to set the GetSFTP processor "Remote Path". What NiFi expression language statement should I use?
Q: GetHTTP & InvokeHTTP - The GetHTTP processor works fine with static filename when getting files from a website. However, I have a use case where I need to download a file daily and the filename is the date of today, ie: 09222015.zip. Since the URL property of the GetHTTP does not support expression language, I cannot do something like http://example.com/${now():format('MMddyyyy')}.zip. Is there a way I can specify the filename dynamically? Or using other processor to make this work. Please advise.
Q: GetTwitter - I am trying to use the GetTwitter processor to pull in tweets with a certain keyword. Do I need to pull in the tweets and then use the RouteOnAttribute processor?
Q: PutHDFS - Is there any way to bypass writing FlowFiles on disk and directly pass those files to HDFS as is? Also, if the files are compressed (zip/gzip), can we store the files on HDFS as uncompressed?
Is there any way to connect to a MapR cluster using the HDFS compatible API?
NOTES
|
Connections
Expand |
---|
Q: If I have FlowFiles stuck in my queue, how can I clear them out?
|
...
Remote Process Groups/Site-to-Site
Expand |
---|
Please add questions about remote process groups/site-to-site here. |
Clustering
Q: How can I send data from one NiFi instance to another?
Q: When setting up site-to-site with a remote process group and networking through firewalls, are the TCP connections one-way, from source node, where the graph is running to the remote process group's node only, or are bidirectional TCP connections required? The reason I ask is I’m encountering problems trying to connect from a data center that has an open outbound firewall, but allows no incoming connections. On the target node, there is no indication in the nifi-app.log of the source node even attempting to connect (not sure if debug logging is required).
|
Clustering
Expand |
---|
Q: I set up a two-node cluster, but after I disconnected one node, did some work in the cluster, and tried to reconnect the second node, I got the error "Failed to connect node to cluster because local flow is different than cluster flow.” I deleted the flow.xml.gz file from the node I’m attempting to add and then restarted it, but I still had the same issue. How do I resolve this?
|
Q: I am evaluating NiFi for a large project to see if NiFi would fit as the main data collector. The project I am working on would require retrieving several hundreds of millions of files per day (hundreds of TB per day) so my first question is how to achieve distribution/clustering with NiFi, if that's possible.
Q: I am running a cluster, and after I tried restarting it, I am seeing errors about trying to do something "while in safe mode". What does that mean?
Q: If we have multiple worker nodes in the cluster, do they partition the work if the source allows partitioning - eg: HDFS, or do all the nodes work on the same data? If the nodes partition the work, then how do they coordinate the work distribution and recovery, etc.? From the documentation it appears that the workers are not aware of each other.
|
Controller Services
Expand |
---|
Please add questions about remote process groups/site-to-site controller services here. |
Reporting Tasks
Expand |
---|
Please add questions about remote process groups/site-to-site reporting tasks here. |