...
Configuration changes must be completed prior to upgrading from version 1 to avoid failures. Application settings and flow design changes should be completed and tested before upgrading to version 2.
Migrating from deprecated components and features requires different steps depending on configured properties. The level of difficulty varies based on the number of properties to be changed and the steps involved to maintain similar behavior.
Features
Migrating deprecated features requires changes to application properties or custom component implementations. The steps involved will be different based on implementation details and deployment use cases.
Legacy and Alternative Sensitive Properties Algorithms
Component | org.apache.nifi.encrypt.PropertyEncryptorBuilder |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-property-encryptor |
Deprecation Type | Allowed Values |
Deprecated Version | 1.21.0 |
Migration Difficulty | LOW |
Migration Type | Property Update |
Migration
The nifi.sensitive.props.algorithm property in the nifi.properties configuration has supported a large number of possible values, including improved algorithms introduced in NiFi 1.14.0.
Starting with NiFi 1.14.0, the default value of the Sensitive Properties Algorithm changed from PBEWITHMD5AND256BITAES-CBC-OPENSSL to NIFI_PBKDF2_AES_GCM_256.
NiFi 1.18.0 added deprecation warnings for all algorithms starting with PBE.
NiFi 1.21.0 added deprecation for the following values using the bcrypt or scrypt algorithms, as well as other values using 128 bit keys.
- NIFI_ARGON2_AES_GCM_128
- NIFI_BCRYPT_AES_GCM_128
- NIFI_BCRYPT_AES_GCM_256
- NIFI_PBKDF2_AES_GCM_128
- NIFI_SCRYPT_AES_GCM_128
- NIFI_SCRYPT_AES_GCM_256
NiFi 2.0.0 supports the following options:
- NIFI_ARGON2_AES_GCM_256
- NIFI_PBKDF2_AES_GCM_256
Running the nifi.sh command supports migrating from a deprecated algorithm to a supported algorithm.
nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256
The Administrator's Guide provides additional details related to setting the Sensitive Properties Algorithm.
ProcessContext encrypt and decrypt methods
Component | org.apache.nifi.processor.ProcessContext |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-api |
Deprecation Type | Interface Method |
Deprecated Version | 1.20.0 |
Migration Difficulty | HIGH |
Migration Type | Implementation redesign |
Migration
The ProcessContext interface provides property and component configuration information to Processors through the onTrigger method. The encrypt and decrypt methods supported enciphering and deciphering component-provided strings using the Sensitive Properties Key and Sensitive Properties Algorithm configured in nifi.properties.
These methods did not provide any persistent storage mechanism and also did not provide any guarantees of compatible behavior when changing application properties. Most components depend on framework capabilities to store and retrieve sensitive property values, obviating the need for these methods.
Custom components should use Property Descriptors with the sensitive field enabled to store and retrieve sensitive property values. Alternative uses of the encrypt and decrypt methods require custom implementation.
SSLContextService createSSLContext methods
Component | org.apache.nifi.ssl.SSLContextService |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-ssl-context-service-api |
Deprecation Type | Interface Method |
Deprecated Version | 1.13.0 |
Migration Difficulty | LOW |
Migration Type | Method invocation changes |
Migration
The SSLContextService interface is a standard Controller Service that supports TLS communication for network services. The createSSLContext methods accepted a Client Authentication argument that did not apply to SSLContext instances. The replacement createContext method provides the same runtime behavior without requiring any arguments. Updating custom code to compile against Apache NiFi 1.13.0 or greater and switching to the createContext method provides compatibility with Apache NiFi 2.0.0.
CRON Scheduling Changes
Component | org.apache.nifi.controller.scheduling.CronSchedulingAgent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-framework-core |
Deprecation Type | Method Argument Values |
Deprecated Version | 1.24.0 |
Migration Difficulty | LOW |
Migration Type | Method argument changes |
Migration
Component run scheduling using cron expression changed from using Quartz to Spring Framework CronExpression for parsing the scheduling period in NIFI-12290.
The new parsing approach does not impact most cron expressions, but does introduce the following changes:
- Removed support for the optional Year field as the seventh field in an expression
- Integer definitions for day of the week span from 0 to 7 instead of 1 to 7
Components can no longer be scheduled to run for a particular year, so any referenced to a seventh field in an expression must be removed. Using named day of the week references such as SUN or MON are preferred over integer references.
ExpressionLanguageScope VARIABLE_REGISTRY
In Apache NiFi 2.0, the ExpressionLanguageScope VARIABLE_REGISTRY has been removed. An alternate expression language scope ENVIRONMENT is now available, allowing for similar configurability via JVM system properties or system environment variables. Further detail on this change may be found in NIFI-12079.
Migration
- Logic of existing flow definitions should be updated to no longer depend on the presence of variables at any process group scope, or in the file registry file. Usage of parameter contexts and parameters is recommended to mitigate any gaps in flow definition logic.
- Existing custom NARs should be rebuilt using the NiFi 2.0 API. Any component properties that provide configurability via VARIABLE_REGISTRY should be adjusted to reference one of the current ExpressionLanguageScope options.
Processors
Migrating deprecated Processors and properties requires flow design changes. The steps required range from setting different property values to replacing one Processor with one or more Processors providing similar capabilities.
The section for each deprecation provides the component class name as well as the Deprecation Type and Migration Difficulty. Each section includes Migration steps and example Flow Definitions with selected configuration properties.
Base64EncodeContent
Component Class | org.apache.nifi.processors.standard.Base64EncodeContent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.20.0 |
Migration Difficulty | LOW |
Migration Type | Component Replacement |
...
Component Class | org.apache.nifi.processors.standard.EncodeContent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
Base64EncodeContent supports a single Mode property that can be set to Encode or Decode.
The standard EncodeContent Processor supports Base64 and other formats for both encoding and decoding.
The EncodeContent Processor provides a direct replacement for Base64EncodeContent using standard properties.
...
The Deprecated Flow Definition includes Base64EncodeContent configured for encoding.
The Migrated Flow Definition replaces Base64EncodeContent with EncodeContent configured with Base64 encoding.
...
.
...
...
Component Class | org.apache.nifi.processors.standard.GetHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.9.0 |
Migration Difficulty | LOW |
Migration Type | Component Replacement |
...
Component Class | org.apache.nifi.processors.standard.InvokeHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
GetHTTP supports a number of configurable properties to retrieve data using the HTTP GET method. The URL property controls the primary behavior of the Processor.
...
The Deprecated Flow Definition includes GetHTTP configured to retrieve the SHA-256 checksum for the NiFi 1.20.0 distribution. The Flow Definition also includes a required SSL Context Service configured with the Java Runtime cacerts Trust Store.
The Migrated Flow Definition includes InvokeHTTP configured with the same URL but without the SSL Context Service, which is not required for public trusted HTTPS services.
GetJMSQueue
Component Class | org.apache.nifi.processors.standard.GetJMSQueue |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.3.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.jms.processors.standard.ConsumeJMS |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-jms-standardprocessors-nar |
Migration
GetJMSQueue supports retrieving messages using the Java Message Service Queue interface to access Apache ActiveMQ.
The Processor GetJMSQueue bundles the ActiveMQ libraries and requires the URL property to specify the JMS server location, and the Destination Name property to specify the JMS queue for retrieval. Additional optional properties support TLS communication and authentication as well as batching.
...
The Deprecated Flow Definition includes GetJMSQueue configured for ActiveMQ running on the localhost address with a specified Destination Name.
The Migrated Flow Definition includes ConsumeJMS configured with a JMSConnectionFactoryProvider using the same JMS broker information. The Migrated Flow Definition includes a JMS Parameters context with a Parameter named JmsClientLibrariesPath referencing the local directory containing the Apache ActiveMQ Artemis libraries.
GetJMSTopic
Component Class | org.apache.nifi.processors.standard.GetJMSTopic |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.3.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.jms.processors.ConsumeJMS |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-jms-processors-nar |
Migration
GetJMSTopic supports retrieving messages using the Java Message Service Topic interface to access Apache ActiveMQ.
GetJMSTopic bundles the ActiveMQ libraries and requires the URL property to specify the JMS server location, and the Destination Name property to specify the JMS topic for retrieval. Additional optional properties support TLS communication and authentication as well as batching.
The standard ConsumeJMS Processor similar functionality to GetJMSTopic, but requires additional configuration. ConsumeJMS does not bundle JMS libraries and requires referencing a local directory containing implementation JAR files.
ConsumeJMS should be configured with a Connection Factory Service implementation of JMSConnectionFactoryProvider with required properties. The ActiveMQ Artemis library provides the current supported version of JMS access.
Property | Value |
---|---|
JMS Connection Factory Implementation Class | org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory |
JMS Client Libraries | /opt/activemq-artemis-2.28.0/lib |
JMS Broker URI | tcp://${AMQ_HOST}:61616 |
ConsumeJMS uses the Connection Factory Service along with several other properties to access JMS Queues.
Property | Value |
---|---|
Connection Factory Service | JMSConnectionFactoryProvider |
Destination Name | |
Destination Type | TOPIC |
Flow Definitions
The Deprecated Flow Definition includes GetJMSTopic configured for ActiveMQ running on the localhost address with a specified Destination Name.
The Migrated Flow Definition includes ConsumeJMS configured with a JMSConnectionFactoryProvider using the same JMS broker information. The Migrated Flow Definition includes a JMS Parameters context with a Parameter named JmsClientLibrariesPath referencing the local directory containing the Apache ActiveMQ Artemis libraries.
GetTwitter
Component Class | org.apache.nifi.processors.twitter.GetTwitter |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-social-media-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.17.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.processors.twitter.ConsumeTwitter |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-social-media-nar |
Migration
GetTwitter uses Version 1 of the Twitter REST API and relies on the retired Twitter Hosebird Client library.
GetTwitter supports several different endpoints for streaming Tweets, including Sample realtime Tweets and Filter realtime Tweets.
GetTwitter also supports configurable properties for filtering Tweets, but this filtering approach is not support in Version 2 of the Twitter REST API, and is not supported in ConsumeTwitter.
ConsumeTwitter uses Version 2 of the Twitter REST API. Twitter documentation provides a general guide for migrating from Version 1 to Version 2 for filtered streams and sampled streams.
ConsumeTwitter with Twitter API Version 2 requires a single OAuth 2.0 Bearer Token for app-only authentication. This approach replaces the API Key and Access Token approach required for GetTwitter.
Configuring ConsumeTwitter with the Sample Stream setting provides similar behavior to GetTwitter Sample Endpoint setting.
Configuring ConsumeTwitter with the Search Stream setting requires creating Filter Rules using the Twitter REST API outside of processor properties.
Twitter Filter Rules
Twitter Filter Rules can be created using JSON sent as an HTTP POST to the Twitter REST API Version 2.
- https://api.twitter.com/2/tweets/search/stream/rules
The Languages property in GetTwitter can be applied to a Filter Rule using the lang operator in a Filter value definition.
{
"add": [
{
"value": "Apache lang:en",
"tag": "English Language Filter"
}
]
}
The Terms to Filter On property in GetTwitter can be applied to a Filter Rule using the Filter value definition with optional Boolean operators.
{
"add": [
{
"value": "Apache AND NiFi",
"tag": "Boolean Terms Filter"
}
]
}
The IDs to Filter On property in GetTwitter can be applied to a Filter Rule using the from operator in a Filter value definition.
{
"add": [
{
"value": "from:1625695877326340102",
"tag": "Twitter User ID Filter"
}
]
}
Flow Definitions
The Deprecated Flow Definition includes GetTwitter configured with the Sample Endpoint and without filters.
The Migrated Flow Definition includes ConsumeTwitter configured with the Sample Stream.
HashAttribute
Component Class | org.apache.nifi.processors.standard.HashAttribute |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.21.0 |
Migration Difficulty | LOW |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.processors.standard. |
UpdateAttribute | |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
HashAttribute supports generating an MD5 digest of one more FlowFile attribute name and value pairs. The MD5 algorithm is not secure for cryptographic operations, and alternatives such as the SHA-2 or SHA-3 algorithms should be used.
The CryptographicHashAttribute UpdateAttribute Processor supports generating a digest of one or more FlowFile attributes using a configurable algorithm.Each Processor uses different approaches for specifying which FlowFile attributes to hashthe hash Expression Language function.
HashAttribute uses the Hash Value Attribute Key property to specify the destination FlowFile attribute name for the computed digest. HashAttribute uses one or more dynamic properties to provide source FlowFile attribute name, and a regular expression value to determine whether the FlowFile attribute should be included in the digest calculation. HashAttribute converts all attribute values to bytes using UTF-8 encoding.
Generating a hash attribute from a single FlowFile attribute name and value pair requires the both standard and dynamic properties.
Property | Value |
---|---|
Hash Value Attribute Key | label-hash |
label | .* |
CryptographicHashAttribute UpdateAttribute uses dynamic properties to specify the source and the destination FlowFile attribute names. The Hash Algorithm property should be configured with SHA-256 or other SHA-2 algorithm.name, and the attribute value as an expression.
Generating a hash attribute from a single FlowFile attribute name and value requires both standard and one dynamic propertiesproperty.
Property | Value |
---|
label-hash | label${label:hash('SHA-256 |
')} |
Flow Definitions
The Deprecated Flow Definition includes HashAttribute configured to write a FlowFile attribute named label-hash using the label attribute name and the value of the attribute from the incoming FlowFile.
The Migrated Flow Definition includes CryptographicHashAttribute configured to write a FlowFile attribute named label-hash using the value of the label attribute from the incoming FlowFile.
HashContent
Component Class | org.apache.nifi.processors.standard.HashContent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.8.0 |
Migration Difficulty | LOW |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.processors.standard.CryptographicHashContent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
HashContent supports generating a digest of FlowFile content using a configurable algorithm and writing the hash to a specified FlowFile attribute.
CryptographicHashContent supports the same basic digest capabilities and writes the hash to a FlowFile attribute name based on the configured algorithm.
The CryptographicHashContent FlowFile attribute name begins with content. When configured with the SHA-256 algorithm, CryptographicHashContent writes a FlowFile attribute named content_SHA-256.
Both Processors have a Hash Algorithm property, but the CryptographicHashContent Processor supports a scoped set of well-defined algorithms instead of depending on the algorithms that the Java Runtime Security Provider enumerates.
Flow Definitions
The Deprecated Flow Definition includes HashContent configured to write a FlowFile attribute named hash.value using the SHA-256 algorithm.
The Migrated Flow Definition includes CryptographicHashContent configured with the SHA-256 algorithm. The Migrated Flow Definition also includes an UpdateAttribute Processor that writes a FlowFile attribute named hash.value using the value of content_SHA-256, providing behavior equivalent to the HashContent Hash Attribute Name property.
InvokeHTTP
Component Class | org.apache.nifi.processors.standard.InvokeHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Properties |
Deprecated Version | 1.18.0 |
Migration Difficulty | LOW |
Migration Type | Property Replacement |
Migration
InvokeHTTP includes several deprecated properties for configuring access through a proxy server.
- Proxy Host
- Proxy Port
- Proxy Type
- Proxy Username
- Proxy Password
InvokeHTTP supports a Controller Service property that abstracts proxy server access.
- Proxy Configuration Service
Creating and configuring an instance of StandardProxyConfigurationService with the corresponding property values is required for upgrading.
Flow Definitions
The Deprecated Flow Definition includes InvokeHTTP configured with Proxy Host and Proxy Port properties.
The Migrated Flow Definition includes InvokeHTTP configured with a StandardProxyConfigurationService containing the same proxy property values.
PostHTTP
Component Class | org.apache.nifi.processors.standard.PostHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.9.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.processors.standard.InvokeHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
PostHTTP supports sending FlowFile content to a configurable HTTP URL using the HTTP POST method.
PostHTTP has a required URL property and also requires an SSL Context Service when the URL indicates an HTTPS location.
PostHTTP supports a specialized mode for transmitting binary packages containing FlowFile attributes and content to the ListenHTTP Processor. This behavior can be enabled using the Send as FlowFile property. There are various approach to implementing similar behavior, including configurable the MergeContent Processor with the FlowFile Version 3 format prior to sending through the InvokeHTTP Processor. Other options include passing selected FlowFile attribute values as HTTP headers, subject to ASCII encoding and length limitations.
The InvokeHTTP Processor supports standard HTTP methods including POST. Unlike PostHTTP, InvokeHTTP does not require an SSL Context Service for accessing HTTPS locations as it defaults to the Java Runtime default certificate authorities.
InvokeHTTP has a required HTTP URL property and also has lower default socket timeout settings than PostHTTP.
Property | Value |
---|---|
HTTP Method | POST |
HTTP URL | HTTP or HTTPS location |
Flow Definitions
The Deprecated Flow Definition includes PostHTTP configured to send to ListenHTTP without custom FlowFile binary packaging.
The Migrated Flow Definition includes InvokeHTTP configured to send to ListenHTTP using equivalent settings.
PutJMS
Component Class | org.apache.nifi.processors.standard.PutJMS |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.3.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.jms.processors.PublishJMS |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-jms-processors-nar |
Migration
PutJMS supports sending messages using the Java Message Service interface to access Apache ActiveMQ.
PutJMS bundles the ActiveMQ libraries and requires the URL property to specify the JMS server location, and the Destination Name property to specify the JMS destination. PutJMS also supports a configurable Destination Type of Queue or Topic.
The PublishJMS Processor supports similar functionality to PutJMS, but requires additional configuration. PublishJMS does not bundle JMS libraries and requires referencing a local directory containing implementation JAR files.
PublishJMS should be configured with a Connection Factory Service implementation of JMSConnectionFactoryProvider with required properties. The ActiveMQ Artemis library provides the current supported version of JMS access.
Property | Value |
---|---|
JMS Connection Factory Implementation Class | org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory |
JMS Client Libraries | /opt/activemq-artemis-2.28.0/lib |
JMS Broker URI | tcp://${AMQ_HOST}:61616 |
PublishJMS uses the Connection Factory Service along with several other properties to access JMS destinations.
Property | Value |
---|---|
Connection Factory Service | JMSConnectionFactoryProvider |
Destination Name | |
Destination Type |
Flow Definitions
The Deprecated Flow Definition includes PutJMS configured for ActiveMQ running on the localhost address with a specified Destination Name.
The Migrated Flow Definition includes PublishJMS configured with a JMSConnectionFactoryProvider using the same JMS broker information. The Migrated Flow Definition includes a JMS Parameters context with a Parameter named JmsClientLibrariesPath referencing the local directory containing the Apache ActiveMQ Artemis libraries.