Version 0.4.0

Version 0.4.0 of Apache NiFi is a provides substantial improvements in functionality and usability, as well as providing some stability and performance improvements and bug fixes.

Highlights of 0.4.0 release include:

• New Application Features
  • Multiple Authentication Mechanisms: NiFi now supports multiple Authentication Mechanisms! No longer is NiFi tied to being either non-secure or security based on two-way SSL but now can provide
    User Authentication via LDAP. This was a significant undertaking, but has paved the way to far more easily provide new Authentication Mechanisms. Future releases
    will include additional mechanisms, such as Kerberos.
  • Drop FlowFiles from Queue: Users are now able to right-click on a Connection and drop the FlowFiles in the queue, rather than relying on FlowFile Expiration to remove unwanted FlowFiles.
• Usability Improvements
  • Explicit Processor Connectivity: Processors that do not expect incoming data will no longer allow incoming Connections. Attempting to draw a connection to a "Source Processor" will show the
    connection line as a red, dotted line, and will not allow the Connection to be made. Likewise, Processors that require input to perform work will be invalid until they have an incoming Connection.
    Some Processors may accept incoming data or run without any incoming data. These Processors will be valid regardless of whether or not they have incoming connections.
  • Getting Started Guide: Getting Started Guide is added to the 'help' screen of the application. This guide provides an introduction to NiFi terms, introduces the key concepts of NiFi and
    discusses how to work with FlowFiles and their Attributes. This is similar in concept to the User Guide but is far less verbose and explains concepts at a higher level.
  • Provenance Fetch Event: A new Provenance Event Type (FETCH) was added. This Event Type is used to indicate that an existing FlowFile's contents were modified as a result of obtaining
    data from an external resource. This is in contrast to a RECEIVE event, which is used to indicate that a FlowFile entered the system as a result of obtaining data from an external resource.
• New Extensions
  
  • RouteText: Allows user to easily establish queries against textual data. Each line of a FlowFile is matched against the specified rules and routed according to the rules (potentially many lines of text
    are included in each output FlowFile). Also supports grouping of textual data so that FlowFiles that are output do not contain text from two different groups.
  • TailFile: Allows user to "tail" a file, consuming data from the end of the file as it is written by another process. This is typically used to consume data from log files as it is written. Processor will pick up
    where it left off, even if NiFi is restarted and log files roll over.
  • ListenSyslog / PutSyslog: Listens for incoming Syslog events over UDP or TCP and sends events to Syslog.
  • ListFile / FetchFile: Performs a listing of files in a given directory and fetches those files. These processors differ from GetFile in that the ListFile processor keeps state about files that have been consumed,
    so that the file can be ingested only once without deleting the source file. Additionally, if the directory being monitored exists in a mounted volume, the state can be shared across the cluster so that a new
    primary node can pick up where the previous processor left off, and the listing can also be shared across the cluster, distributing the work of pulling in and processing the files.
  • ListSFTP / FetchSFTP: Performs a listing of files on an SFTP server and fetches those files. Similarly to ListFile / FetchFile, state can be distributed across the cluster so that multiple nodes can perform the
    work in parallel.
  • DeleteS3Object: Removes an Object from Amazon S3.
  • PutHBaseCell / GetHBase / PutHBaseJSON: Allows users to put the contents of a FlowFile to HBase and listen for changes to an HBase table, automatically pulling in the rows that are added/updated.
  • GetAzureEventHub / PutAzureEventHub: Send the contents of FlowFiles as Events to Microsoft Azure Event Hub or listen for incoming Events on an Event Hub and create FlowFiles for those Events.
  • GetCouchbaseKey / PutCouchbaseKey: Send the contents of FlowFiles to Couchbase or fetch the contents of a record from Couchbase.
  • AttributesToJSON: Easily form a JSON document (as the contents of a FlowFile or as an Attribute) from a user-defined set of FlowFile Attributes.
  • SplitAvro: Splits a FlowFile that consists of many Avro records into individual FlowFiles, each containing a smaller number of Avro records.
  • ExtractAvroMetadata: Extracts the metadata from the header of an Avro file and adds the metadata to the FlowFile as a set of Attributes.
  • Image Content Viewer: When users look at the details of a Provenance Event, in the Content tab, if the View button is clicked, and the contents of the FlowFile are an image, that image will be rendered in
    the UI, rather than indicating that no viewer is available for this content type.

A full list of issues that were resolved can be found at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12333070

Version 0.3.0

Highlights of 0.3.0 release include

...