...

• When moving between patch (also known as incremental) version changes such as 0.1.0 to 0.1.1 users should be safe to assume a clean upgrade can occur with no risk of behavior changes other than bug fixes and no compatibility issues.
• When moving between minor changes such as 0.1.0 to 0.2.0 users can expect new behaviors and bug fixes but backward compatibility should be protected.
• When moving between major changes such as 0.x.y to 1.0.0 t here may be backward compatibility impacting changes largely focused on removal of deprecated items

Versions

Migrating to 2.0.0-M2

Migrating

...

to 2.

...

0.0-M2 requires upgrading to 1.

...

25.0 prior to moving to 2.0.0-M2.

Version 2.0.0-M2 is the second milestone version of Apache NiFi 2.0.0.

Please refer to the Deprecated Components and Features documentation for items removed in this version.

In addition to the changes from 2.0.0-M1, version 2.0.0-M2 includes several additional changes that may require manual adjustments when upgrading.

Deployment and Configuration Changes

• JoltTransformJSON and JoltTransformRecord moved to new nifi-jolt-nar
• Removed InfluxDB Processors
• Removed Bootstrap Notification Services
• Removed MiNiFi C2 Server
• Moved from SimpleDateFormat to DateTimeFormatter for date and time parsing and formatting

Custom Component Changes

• Custom Components with Property Descriptor Builders passing null to the defaultValue() method are no longer supported
• Upgrading from Servlet API 3 to Servlet API 6 requires changes from javax.servlet packages to jakarta.servlet packages

Migrating to 2.0.0-M1

Migrating to 2.0.0-M1 requires upgrading to 1.24.0 prior to moving to 2.0.0-M1.

Version 2.0.0-M1 is the first milestone version of Apache NiFi 2.0.0. This version includes over 900 issues resolved, including new features as well as numerous improvements and bug fixes.

Please refer to the Deprecated Components and Features documentation for items removed in this version.

Migrating Deprecated Components and Features for 2.0.0 includes current progress on the steps necessary for migrating certain components and features.

• The nifi.flow.configuration.file property in nifi.properties must be changed to reference flow.json.gz instead of flow.xml.gz
• The default bootstrap configuration no longer includes java.net.preferIPv4Stack=true as part of the standard settings. Systems that support IPv6 may require adjustments to host resolution settings.
• The ExpressionLanguageScope VARIABLE_REGISTRY has been removed.  An alternate expression language scope ENVIRONMENT is now available, allowing for similar configurability via JVM system properties or system environment variables.

Migrating to 1.24.0

• URL properties specified in components should be URL-encoded. NIFI-12103 NIFI-12513
  • InvokeHTTP (property HTTP URL)

Migrating to 1.23.0

• The resolution for CVE-2023-36542 included the addition of a new Access Policy component restriction named Reference Remote Resources applied to the following components:
  • CaptureChangeMySQL
  • ConsumeJMS
  • DBCPConnectionPool
  • ExtractGrok
  • GrokReader
  • HikariCPConnectionPool
  • HadoopDBCPConnectionPool
  • HBase_2_ClientService
  • JMSConnectionFactoryProvider
  • PublishJMS
  • ValidateJson
  • ValidateXml
• Deployments with fine-grained Access Policies may require assigning the Reference Remote Resources permission to users who should be able to configure applicable components
• The nifi-riemann-nar and nifi-hbase_1_1_2-client-service-nar are no longer included in the standard binary, but can be downloaded from Maven Central
• RethinkDB Processors are deprecated for removal in 2.0.0
• Upgrading dependencies from H2 Database Engine version 2.1 to 2.2 required internal automated migration

Migrating to 1.22.0

• DBCPConnectionPool and HikariCPConnectionPool Controller Services no longer support H2 database connections according to Database URL validation added in NIFI-11653
• JndiJmsConnectionFactoryProvider Controller Service along with ConsumeJMS and PublishJMS Processors no longer support LDAP Provider URLs according to validation added in NIFI-11614
  • The additional details documentation for JndiJmsConnectionFactoryProvider describes an optional Java System property that can be added to bootstrap.conf to enable LDAP for specific use cases
• GetAzureQueueStorage and PutAzureQueueStorage have been deprecated in favor of GetAzureQueueStorage_v12 and PutAzureQueueStorage_v12
• ECMAScript, lua, and ruby have been deprecated for Scripted components in favor of Groovy
• GetIgniteCache and PutIgniteCache have been deprecated for removal in 2.0.0
• The Compression Type property has been removed from AbstractCassandraProcessor which affects PutCassandraQL, PutCassandraRecord, and QueryCassandra. This will cause existing instances of these processors to become invalid. To migrate the flow, remove that property which is now displayed as an unsupported property in the processors' configuration dialogs. If a Compression Type other than the default "NONE" is needed, use a Cassandra Connection Provider controller service rather than the direct properties in the processor. (NIFI-5642)

Migrating to 1.21.0

• Refactored OpenID Connect integration changed the Post Logout Redirect URI to avoid using relative paths
  • Previous Post Logout Redirect URI Path: /nifi-api/../nifi/logout-complete
  • New Post Logout Redirect URI Path: /nifi/logout-complete
  • OpenID Connect Identity Providers that support RP-Initiated Logout 1.0 may need to be updated with the new Post Logout Redirect URI
• Refactored OpenID Connect integration changed application Bearer Token expiration to derive from the Access Token instead of the ID Token
  • Some Identity Providers return the same expiration for both Access Tokens and ID Tokens, but providers that support different values may need to be updated so that the Access Token reflects the expected application session expiration

Migrating to 1.19.0

• This release deprecated support for Java 8. Java 11.0.16 is the minimum recommended version
• For users of the apace/nifi* Docker images, note that the base image changed to eclipse-temurin:jre-11 from openjdk:8-jre. This could cause build issues if you were using the Apache NiFi docker image(s) as a base for custom images, or otherwise have functionality that requires packages from the base image and not NiFi itself. It is recommended to test in a dev environment prior to migrating to 1.19.0/latest image tags.

Migrating to 1.17.0

• This release deprecated support for RocksDBFlowFileRepository. The RocksDB repository is packaged in a separate NAR named nifi-rocksdb-nar, available for download from Maven Central
• This release removed Hive 1.2 components from the standard release binary. The nifi-hive-nar and nifi-hive-services-api-nar modules can be downloaded from Maven Central
• This release upgraded Apache Avro to version 1.11.0 (NIFI-7234). This version of Avro changes the behavior where you could previously have aliases for fields with a null value (AVRO-2278), and thereby introduces a bug in NiFi that will be resolved in NiFi 1.19.0 (NIFI-10660).

Migrating from 1.15.3 and earlier to 1.17.0 or 1.18.0

• Versions 1.15.3 and earlier stored the Flow Configuration in flow.xml.gz, and version 1.16.0 introduced a new serialization format stored in flow.json.gz. As a result of changes in version 1.17.0 to support Sensitive Dynamic Properties, components can be marked as invalid when the Flow Configuration contains a Parameter Context with a Sensitive Parameter Value that is referenced in a Sensitive Property. This issue can be resolved by entering the Sensitive Parameter Value in the referenced component Sensitive Property, which will save the Flow Configuration using the new flow.json.gz format. As an alternative, first upgrading to version 1.16.3 and then upgrading to 1.17.0 also avoids invalid component status issues. This issue was resolved under NIFI-10567 released in version 1.19.0, so upgrading from 1.15.3 to 1.19.0 and following are not subject to this problem.

Migrating from 1.15.0 to 1.16.0

• This release adds HTTP request logging and updates the default logback.xml configuration. Deployments with customized Logback configurations should add a new request appender and request logger configuration to avoid writing HTTP requests to nifi-app.log
• This release removed support for Elasticsearch 2 components. Version 1.15.3 of nifi-elasticsearch-nar can be downloaded from Maven Central
• This release removed support for Elasticsearch 5 components. Version 1.15.3 of nifi-elasticsearch-5-nar can be downloaded from Maven Central
• This release removed support for Kite components. Version 1.15.3 of nifi-kite-nar can be downloaded from Maven Central
• This release removed support for Lumberjack components. Version 1.15.3 of nifi-lumberjack-nar can be downloaded from Maven Central
• This release removed support for Kafka components supporting Kafka versions prior to 1.0. Version 1.15.3 of nifi-kafa-0-nar modules can be downloaded from Maven Central
• This release removed support for InfluxDB from the standard NiFi binary download. The nifi-influxdb-nar can be downloaded from Maven Central
• This release removed the nifi-processor-utils JAR and refactored classes into several new modules under nifi-extension-utils. When rebuilding custom components to depend on 1.16.0 libraries, it will be necessary to remove dependencies on nifi-processor-utils. Custom NAR bundles, already compiled using versions prior to 1.16.0, can continue to be used in runtime deployments
• This release removed support for MySQL 5.6 /5.7 and Postgres 9.x for NiFi Registry's database, supported databases are now H2, Postgres 10.x - 14.x and MySQL 8.x.
• NOTE: A bug was discovered after the release (NIFI-9836) that causes NiFi Registry to only work with H2. The issue is addressed in main and will be resolved in 1.16.1.
• This release adds HTTP request logging and updates the default logback.xml configuration. Deployments with customized Logback configurations should add a new request appender and request logger configuration to avoid writing HTTP requests to nifi-app.log
• This release removed support for Elasticsearch 2 components. Version 1.15.3 of nifi-elasticsearch-nar can be downloaded from Maven Central
• This release removed support for Elasticsearch 5 components. Version 1.15.3 of nifi-elasticsearch-5-nar can be downloaded from Maven Central
• This release removed support for Kite components. Version 1.15.3 of nifi-kite-nar can be downloaded from Maven Central
• This release removed support for Lumberjack components. Version 1.15.3 of nifi-lumberjack-nar can be downloaded from Maven Central
• This release removed support for Kafka components supporting Kafka versions prior to 1.0. Version 1.15.3 of nifi-kafa-0-nar modules can be downloaded from Maven Central
• This release removed the nifi-processor-utils JAR and refactored classes into several new modules under nifi-extension-utils. Custom components that previously depended on nifi-processor-utils can be changed to depend on nifi-utils

Migrating from 1.14.0 to 1.15.0

...

• PutKudu processor - while NIFI-6551 fixes flows writing to Kudu timestamp (UNIXTIME_MICROS) columns via timestamp or date fields. This change could break PutKudu processors that are writing to Kudu timestamp (UNIXTIME_MICROS) columns via numeric fields. Before this change, flows would often multiply millisecond values by 1000 to write microsecond values to Kudu. On upgrade this multiplication should be removed and milliseconds should be sent.
• HandleHttpRequest was updated to no longer write the 'http.param.*' attributes. These attributes were undocumented and therefore not part of its 'contract'. They were removed because in the case of query parameters, they were duplicative and in the case of multipart/form data, they were both duplicative of FlowFile content and also dangerous as could be potentially extremely large. Multipart/form data was never intended to be included as attributes but were inadvertently included. This is no longer the case. If multipart/form data is needed as an attribute, the appropriate attributes can be captured via the ExtractText processor.
• NARs for Kafka 0.9 and Kafka 0.10 have been removed from the convenience binary. You can still get them from the various artifact repositories and use them in your flows but we cannot bundle them due to space limitations by default.
• The SSLContextService interface was updated as part of NIFI-7407.  Custom NARs that depend on nifi-standard-services-api-nar may need to be rebuilt against version 1.12.0 to ensure compatibility.  Users of those custom nars could also just pull in the appropriate service API and service implementation nars in their environment.
• The underlying implementation for CSVRecordLookupService's "CSV Format" property was changed in NIFI-7770, rendering existing CSVRecordLookupService instances invalid. The dynamic property should be removed and its value set in the "new" CSV Format property. NOTE: This change was applied in 1.12.1, so it does not exist in 1.12.0.

Migrating from 1.x.x to 1.11.x

...

• Kerberos System Properties

  • SPNEGO and service principals for Kerberos are now established via separate system properties.
    • New SPNEGO properties
      • nifi.kerberos.spnego.principal
      • nifi.kerberos.spnego.keytab.location
      • nifi.kerberos.spnego.authentication.expiration
    • New service properties
      
      • nifi.kerberos.service.principal
        
      • nifi.kerberos.service.keytab.location
    • Removed properties
      • nifi.kerberos.keytab.location
        
      • nifi.kerberos.authentication.expiration
• DBCPConnectionPool Service
  
  • The “Database Driver Jar Url” property has been replaced by the “Database Driver Location(s)” property which accepts a comma-separated list of URLs or local files/folders containing the driver JAR.
    
  • Existing processors that reference this service will be invalid until the new property is configured.

• MonitorDiskUsage

  • This standard reporting task has been simplified to let the user specify a logical name, a directory and a threshold to monitor.  Previously it was tightly coupled to the internal flow file and content repositories in a manner that didn't align to the pluggable nature of those repositories.  The new approach gives the user total control over what they want it to monitor.
• Connection/Relationship Default Back Pressure Settings
  • It used to be that by default no backpressure settings were supplied.  This too often meant people learned the value of backpressure the hard way.  New connections made will now have a default value set of 10,000 flowfiles and 1GB worth of data size.

• Multi-tenant Authorization Model

  • Authority Provider model has been replaced by a Multi-tenant Authorization model. Access privileges are now defined by policies that can be applied system-wide or to individual components. Details can be found in the ‘Admin Guide’ under ‘Multi-tenant Authorization’.

  • The system properties nifi.authority.provider.configuration.file and nifi.security.user.authority.provider have been replaced by nifi.authorizer.configuration.file and nifi.security.user.authorizer, respectively. Details on configuration can be found in the “Admin Guide’ under ‘Authorizer Configuration’.

  • 0.7.0 authorized users/roles can be converted to the new authorization model. An existing authorized-users.xml file can be referenced in the authorizers.xml "Legacy Authorized Users File” property to automatically generate users and authorizations. Details on configuration can be found in the “Admin Guide” under ‘Authorizers.xml Setup’.

  • Controller Services that will be used by Processors must be defined in the Operate Palette of the root process group or sub process group where they will be used.  Controller Services defined in the Global - Controller Settings window can only be used by Reporting Tasks, not by any Processors.

• HTTP(S) Site-to-Site

  • HTTP(S) protocol is now supported in Site-to-Site as an underlying transport protocol.

  • HTTP(S) protocol is enabled by default (nifi.remote.input.http.enabled=true).Configuration details can be found in the 'Site-to-Site Properties' section of the 'Admin Guide’. Of note:

    • With both socket and HTTP protocols supported, nifi.remote.input.socket.host has been renamed to nifi.remote.input.host

    • nifi.remote.input.secure is now set to false by default

• Zero-Master Clustering

...