Page History

NiFi employs a Write-Ahead Log to keep track of changes to FlowFiles (i.e., a data record) as they flow through the system. This  This Write-Ahead Log keeps track of changes to the FlowFiles themselves, such as the FlowFile's attributes (key/value pairs that make up metadata), as well as their state, such as which Connection/Queue the FlowFile belongs in.

...

Here, we will describe the implementation details and algorithm that are used in order to implement this capability. Currently,these  these are simply notes that I created in order to help explain the code to someone else, and they serve very well to augment the code. On their own,they  they may not provide a great, holistic explanation of the class. This should eventually be turned into an architectural type of paper or at least a blogpostblog post. For now, though, I will post the notes that I have, so that they can help others to understand the logic as well. 

Definition of Terms

• SerDe: interface to Serialize/Deserialize records and updates to records
• TransactionID Generator: An AtomicLong used to indicate a Transaction ID when writing to edit log or snapshot for each Transaction

...

...

Writing to the Write-Ahead Log

...