Maven Projects for Custom Processors

Overview

Apache NiFi (incubating) processors are typically organized in processor bundles. A processor bundle is generally composed of the following:

• A Maven project that produces a jar of processors
• A Maven project that packages the processors into a NAR
• Parent pom for the bundle that builds the processor and NAR projects

An example processor bundle from the NiFi distribution is the nifi-kafka-bundle, containing the sub-projects nifi-kafka-processors and nifi-kafka-nar.

Maven Archetype

NiFi provides a Maven archetype for easily creating this project structure. To use the archetype, run the following Maven command:

mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.1.0-incubating -DnifiVersion=0.1.0-incubating

The archetypeVersion corresponds to the version of the nifi-processor-bundle-archetype that is being used, currently this version is the same as the top-level NiFi version (i.e. for the NiFi 0.2.0-incubating release, the archetypeVersion would be 0.2.0-incubating, and so on). The nifiVersion is the version of NiFi that the new processors will depend on.

Running the above command will prompt you for the following properties:

Processor Bundle Structure

A processor bundle created with the archetype will create the following project structure:

nifi-basename-bundle

    ├── nifi-basename-nar

    │   └── pom.xml

    ├── nifi-basename-processors

    │   ├── pom.xml

    │   └── src

    │       ├── main

    │       │   ├── java

    │       │   │   └── org.apache.nifi.processors.basename

    │       │   │       └── MyProcessor.java

    │       │   └── resources

    │       │       ├── META-INF

    │       │       │   └── services

    │       │               └── org.apache.nifi.processor.Processor

    │       └── test

    │           └── java

    │               └── org.apache.nifi.processors.basename

    │                   └── MyProcessorTest.java

    └── pom.xml

The default processor generated by the archetype is called MyProcessor. If you decide to rename this processor, make sure to also update the org.apache.nifi.processor.Processor file. This file is used by NiFi to locate available processors, and the fully qualified class name of the processor must be listed here in order for it to be available from the user interface.

Inheritance

By default, the newly generated bundle will inherit from the nifi-nar-bundles artifact from Apache NiFi, for the given nifiVersion that was specified. This is well suited for bundles that plan to be included with Apache NiFi, or plan to be Apache licensed, but it may not be desired in all cases. If it is preferred to not inherit from the nifi-nar-bundles artifact, then the newly generated bundle can be modified in the following ways:

1. Remove the <parent>...</parent> from the bundle's root pom, or replace it with another desired parent
   
   

2. Add the nar plugin to bundle's root pom (ensure the latest version of the nar plugin is being used):

   <build>

           <plugins>

               <plugin>

                   <groupId>org.apache.nifi</groupId>

                   <artifactId>nifi-nar-maven-plugin</artifactId>

                   <version>1.0.1-incubating</version>

                   <extensions>true</extensions>

               </plugin>

           </plugins>

       </build>

3. Specify the necessary dependency versions in the bundle's processors pom. This is required because the default behavior is for the dependencies to inherit their versions through the nifi-nar-bundle parent, so with a different parent these versions will have to be explicitly set. In addition these dependencies should be changed to provided since they will be provided by NiFi.

Common Dependencies

NiFi provides a mechanism to share common functionality through the concept of ControllerServices. A ControllerService is typically composed of the following:

• A Maven project providing the API (i.e. the Java interface that extends the ControllerService interface)
• A Maven project providing an implementation of the above interface
• A Maven project that packages the implementation in a NAR

To leverage a ControllerService a dependency on the appropriate API artifact would be required (the NAR and implementation would be provided at runtime by NiFi). As an example, to leverage the SSLContextService the following dependency would be added to the pom of the processors jar project:

<dependency>

            <groupId>org.apache.nifi</groupId>

            <artifactId>nifi-ssl-context-service-api</artifactId>

</dependency>

This allows a processor in the given project to define the SSLContextService as follows:

public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new PropertyDescriptor.Builder()

            .name("SSL Context Service")

            .description("The Controller Service to use in order to obtain an SSL Context")

            .required(false)

            .identifiesControllerService(SSLContextService.class)

            .build();

An instance of the service can then be obtained:

final SSLContextService sslContextService = context.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);

 The nifi-standard-services module contains the shared APIs that can be leveraged. Browsing this module can provide the groupId, artifactId, and version of the various APIs.