Page History

Kerberos System Properties

• SPNEGO and service principals for Kerberos are now established via separate system properties.
  • New SPNEGO properties
    • nifi.kerberos.spnego.principal
    • nifi.kerberos.spnego.keytab.location
    • nifi.kerberos.spnego.authentication.expiration
  • New service properties
    
    • nifi.kerberos.service.principal
      
    • nifi.kerberos.service.keytab.location
  • Removed properties
    • nifi.kerberos.keytab.location
      
    • nifi.kerberos.authentication.expiration

• The “Database Driver Jar Url” property has been replaced by the “Database Driver Location(s)” property which accepts a comma-separated list of URLs or local files/folders containing the driver JAR.
  
• Existing processors that reference this service will be invalid until the new property is configured.

MonitorDiskUsage

• This standard reporting task has been simplified to let the user specify a logical name, a directory and a threshold to monitor.  Previously it was tightly coupled to the internal flow file and content repositories in a manner that didn't align to the pluggable nature of those repositories.  The new approach gives the user total control over what they want it to monitor.

• It used to be that by default no backpressure settings were supplied.  This too often meant people learned the value of backpressure the hard way.  New connections made will now have a default value set of 10,000 flowfiles and 1GB worth of data size.

Multi-tenant Authorization Model

• Authority Provider model has been replaced by a Multi-tenant Authorization model. Access privileges are now defined by policies that can be applied system-wide or to individual components. Details can be found in the ‘Admin Guide’ under ‘Multi-tenant Authorization’.

• The system properties nifi.authority.provider.configuration.file and nifi.security.user.authority.provider have been replaced by nifi.authorizer.configuration.file and nifi.security.user.authorizer, respectively. Details on configuration can be found in the “Admin Guide’ under ‘Authorizer Configuration’.

• 0.7.0 authorized users/roles can be converted to the new authorization model. An existing authorized-users.xml file can be referenced in the authorizers.xml "Legacy Authorized Users File” property to automatically generate users and authorizations. Details on configuration can be found in the “Admin Guide” under ‘Authorizers.xml Setup’.

HTTP(S) Site-to-Site

• HTTP(S) protocol is now supported in Site-to-Site as an underlying transport protocol.

• HTTP(S) protocol is enabled by default (nifi.remote.input.http.enabled=true).Configuration details can be found in the 'Site-to-Site Properties' section of the 'Admin Guide’. Of note:

  • With both socket and HTTP protocols supported, nifi.remote.input.socket.host has been renamed to nifi.remote.input.host

  • nifi.remote.input.secure is now set to false by default

Zero-Master Clustering

• Master/slave clustering model has been replaced by a Zero-Master Clustering paradigm.  Each node in a NiFi cluster performs the same tasks on the data, but each operates on a different set of data.  A DataFlow manager can now interact with the NiFi cluster through the UI of any node.

• ZooKeeper elects a single node as the Cluster Coordinator and also handles failover. All cluster nodes report heartbeat and status information to the Cluster Coordinator, which is responsible for disconnecting and connecting nodes. Additionally, every cluster has one Primary Node, also elected by ZooKeeper.

• Configuration details can be found in the ‘Clustering Configuration’ section of the ‘Admin Guide’ as well as the Cluster Common/Node and ZooKeeper Properties section of the ‘Admin Guide’.  Of note:

• • • NiFi Cluster Manager (NCM) configuration and properties are no longer relevant and have been removed.

    • The following properties should be set on each node:

      • nifi.web.http.port=<node port>

      • nifi.cluster.is.node=true

      • nifi.cluster.node.address=<fully qualified hostname of the node>

      • nifi.cluster.node.protocol.port=<node protocol port>

      • nifi.state.management.embedded.zookeeper.start=true

      • nifi.state.management.provider.cluster=zk-provider

      • nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties

      • nifi.zookeeper.connect.string=<A comma-separated list of host:port pairs to connect to ZooKeeper. For example, my-zk-server1:2181,my-zk-server2:2181,my-zk-server3:2183>

• • Embedded ZooKeeper setup

    • The zookeeper.properties file needs to be populated with a list of each node's embedded ZooKeeper server. The servers are specified in the form of server.1, server.2, to server.n. Each of these servers is configured as <hostname>:<quorum port>[:<leader election port>]. For example, server.1=nifi-node1-hostname:2888:3888.

    • The zookeeper.properties file has a property named dataDir which is set to ./state/zookeeper by default. For each node, create a file named myid and place it in this directory. The contents of this file should be the index of the server as specified by the server.<number>.  Configuration details can be found in the “Admin Guide’ under ‘Embedded ZooKeeper Server’.

  • State Management

  • • In the state-management.xml file, set the “Connect String” property to the same list of ZooKeeper host:port pairs used for the nifi.zookeeper.connect.string property value.

  • Secure Clustered Environment

  • • The identities for each node must be specified in the authorizers.xml file. The authorization policies required for the nodes to communicate will then be created during startup.  Details on configuration can be found in the ‘Admin Guide’ under ‘Authorizers.xml Setup’.

Secured Zookeeper

• The username and password mechanism to provide ZooKeeper authentication is no longer supported.  As a result, the “Username” and “Password” properties in the state-management.xml file have been removed.

• The “Access Control” property in the state-management.xml file is now set to “Open” by default.  It should be changed to “CreatorOnly” when Zookeeper is secured via Kerberos.

QueryDatabaseTable Processor

• The 'SQL Pre-processing Strategy' property has been replaced by the 'Database Type’ property.  This property sets the type of database for generating database-specific code. Property values include ‘Generic' (default) and ‘Oracle’ (for custom SQL clauses).

TailFile Processor

• TailFile originally stored state in a local file, then state management was added in 0.5.0 to support reading in the local state and moving it into the state manager.  In 1.0.0, the auto-migration from the old state mechanism has been removed.

• If upgrading from a pre-0.5.0 version of NiFi, it is suggested to upgrade to a version greater than or equal to 0.5.0 first, then go to 1.0.0 to not lose state on existing TailFile processors.

LDAP referral strategy ‘IGNORE’ bug fix

• Errors occurred if the Referral Strategy was set to ‘IGNORE’ due to a typo in the code to accept ‘INGORE' instead.  The login-identity-providers.xml file should now be configured with the intended value of ‘IGNORE’ for the Referral Strategy property.