Q: I set up a two-node cluster, but after I disconnected one node, did some work in the cluster, and tried to reconnect the second node, I got the error "Failed to connect node to cluster because local flow is different than cluster flow.” I deleted the flow.xml.gz file from the node I’m attempting to add and then restarted it, but I still had the same issue. How do I resolve this?
- A: This problem can occur if you have templates on the second node that are not in sync with the cluster. On the second node, delete the flow.xml.gz and also delete the templates (located by default in conf/templates), then try to restart the node and see if it can connect to the cluster.
Q: I am evaluating NiFi for a large project to see if NiFi would fit as the main data collector. The project I am working on would require retrieving several hundreds of millions of files per day (hundreds of TB per day) so my first question is how to achieve distribution/clustering with NiFi, if that's possible. - A: The NiFi Administrator Guide includes a quick explanation of our clustering capabilities and example configurations. This would be a great place to start and become familiar with NiFi clustering.
Q: I am running a cluster, and after I tried restarting it, I am seeing errors about trying to do something "while in safe mode". What does that mean? - A: When you restart a cluster, it will stay in safe mode until your Primary Node connects. Here is an excerpt from the Admin Guide that explains how this works and how you can adjust it: "If the cluster restarts, the NCM will "remember" which node was the Primary Node and wait for that node to re-connect before allowing the DFM to make any changes to the dataflow. The ADMIN may adjust how long the NCM waits for the Primary Node to reconnect by adjusting the property nifi.cluster.manager.safemode.duration in the nifi.properties file." By default, that property is set to wait forever for the Primary Node to connect, but if you do not care which node is the primary node, you can set a time limit here, and the NCM will automatically elect a new Primary Node after that amount of time has passed.
Q: If we have multiple worker nodes in the cluster, do they partition the work if the source allows partitioning - eg: HDFS, or do all the nodes work on the same data? If the nodes partition the work, then how do they coordinate the work distribution and recovery, etc.? From the documentation it appears that the workers are not aware of each other. - A: Nodes in a cluster work independently from one another and do not know about each other. Each node in a cluster runs the same flow. Typically, if you want to pull from HDFS and partition that data across the cluster, you would run ListHDFS on the Primary Node only, and then use site-to-site (remote process group) to distribute that listing to all nodes in the cluster. Each node would then pull the data that it is responsible to pull and begin working on it. We do realize that this is not ideal to have to setup this way, and it is something that we are working on so that it is much easier to have that listing automatically distributed across the cluster.
|