Page History

NiFi employs a Write-Ahead Log to keep track of changes to FlowFiles (i.e., a data record) as they flow through the system.  This Write-Ahead Log keeps track of changes to the FlowFiles themselves, such as the FlowFile's attributes (key/value pairs that make up metadata), as well as their state, such as which Connection/Queue the FlowFile belongs in.

Here, we will describe the implementation details and algorithm that are used in order to implement this capability. Currently, these are simply notes that I created in order to help explain the code to someone else, and they serve very well to augment the code. On their own, they may not provide a great, holistic explanation of the class. This should eventually be turned into an architectural type of paper or at least a blog post. For now, though, I will post the notes that I have, so that they can help others to understand the logic as well.

...