...
Migrating from deprecated components and features requires different steps depending on configured properties. The level of difficulty varies based on the number of properties to be changed and the steps involved to maintain similar behavior.
...
Features
Migrating deprecated Processors and properties features requires flow design changeschanges to application properties or custom component implementations. The steps required range from setting different property values to replacing one Processor with one or more Processors providing similar capabilities.
The section for each deprecation provides the component class name as well as the Deprecation Type and Migration Difficulty. Each section includes Migration steps and example Flow Definitions with selected configuration properties.
Base64EncodeContent
involved will be different based on implementation details and deployment use cases.
Legacy and Alternative Sensitive Properties Algorithms
ComponentComponent Class | org.apache.nifi.processorsencrypt.standard.Base64EncodeContentPropertyEncryptorBuilder |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standardproperty-narencryptor |
Deprecation Type | Component ClassAllowed Values |
Deprecated Version | 1.2021.0 |
Migration Difficulty | LOW |
Migration Type | Component ReplacementProperty Update |
Alternatives
...
org.apache.nifi.processors.standard.EncodeContent
...
Migration
Base64EncodeContent supports a single Mode property that can be set to Encode or Decode.
The EncodeContent Processor supports Base64 and other formats for both encoding and decoding.
EncodeContent provides a direct replacement for Base64EncodeContent using standard properties.
...
Flow Definitions
The Deprecated Flow Definition includes Base64EncodeContent configured for encoding.
The Migrated Flow Definition replaces Base64EncodeContent with EncodeContent configured with Base64 encoding.
GetHTTP
...
org.apache.nifi.processors.standard.GetHTTP
...
Migration
The nifi.sensitive.props.algorithm property in the nifi.properties configuration has supported a large number of possible values, including improved algorithms introduced in NiFi 1.14.0.
Starting with NiFi 1.14.0, the default value of the Sensitive Properties Algorithm changed from PBEWITHMD5AND256BITAES-CBC-OPENSSL to NIFI_PBKDF2_AES_GCM_256.
NiFi 1.18.0 added deprecation warnings for all algorithms starting with PBE.
NiFi 1.21.0 added deprecation for the following values using the bcrypt or scrypt algorithms, as well as other values using 128 bit keys.
- NIFI_ARGON2_AES_GCM_128
- NIFI_BCRYPT_AES_GCM_128
- NIFI_BCRYPT_AES_GCM_256
- NIFI_PBKDF2_AES_GCM_128
- NIFI_SCRYPT_AES_GCM_128
- NIFI_SCRYPT_AES_GCM_256
NiFi 2.0.0 supports the following options:
- NIFI_ARGON2_AES_GCM_256
- NIFI_PBKDF2_AES_GCM_256
Running the nifi.sh command supports migrating from a deprecated algorithm to a supported algorithm.
nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256
The Administrator's Guide provides additional details related to setting the Sensitive Properties Algorithm.
ProcessContext encrypt and decrypt methods
Component |
Alternatives
Component Class | org.apache.nifi.processorsprocessor.standard.InvokeHTTPProcessContext |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
GetHTTP supports a number of configurable properties to retrieve data using the HTTP GET method. The URL property controls the primary behavior of the Processor.
GetHTTP requires the Filename property to set the corresponding FlowFile attribute for content retrieved. The Processor requires an SSL Context Service when the URL starts with the HTTPS scheme.
The standard InvokeHTTP Processor supports standard HTTP methods including GET. Unlike GetHTTP, InvokeHTTP does not require an SSL Context Service for accessing HTTPS locations as it defaults to the Java Runtime default certificate authorities.
InvokeHTTP has a required HTTP URL property and defaults to setting the FlowFile filename attribute to a random UUID. The filename attribute can be derived from the URL path using the Response FlowFile Naming Strategy property set to URL_PATH.
InvokeHTTP has lower default socket timeout settings than GetHTTP.
Setting standard InvokeHTTP properties provides functionality similar to GetHTTP,
...
api | |
Deprecation Type | Interface Method |
Deprecated Version | 1.20.0 |
Migration Difficulty | HIGH |
Migration Type | Implementation redesign |
Migration
The ProcessContext interface provides property and component configuration information to Processors through the onTrigger method. The encrypt and decrypt methods supported enciphering and deciphering component-provided strings using the Sensitive Properties Key and Sensitive Properties Algorithm configured in nifi.properties.
These methods did not provide any persistent storage mechanism and also did not provide any guarantees of compatible behavior when changing application properties. Most components depend on framework capabilities to store and retrieve sensitive property values, obviating the need for these methods.
Custom components should use Property Descriptors with the sensitive field enabled to store and retrieve sensitive property values. Alternative uses of the encrypt and decrypt methods require custom implementation.
SSLContextService createSSLContext methods
Component | org.apache.nifi.ssl.SSLContextService |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-ssl-context-service-api |
Deprecation Type | Interface Method |
Deprecated Version | 1.13.0 |
Migration Difficulty | LOW |
Migration Type | Method invocation changes |
Migration
The SSLContextService interface is a standard Controller Service that supports TLS communication for network services. The createSSLContext methods accepted a Client Authentication argument that did not apply to SSLContext instances. The replacement createContext method provides the same runtime behavior without requiring any arguments. Updating custom code to compile against Apache NiFi 1.13.0 or greater and switching to the createContext method provides compatibility with Apache NiFi 2.0.0.
CRON Scheduling Changes
Component |
Flow Definitions
The Deprecated Flow Definition includes GetHTTP configured to retrieve the SHA-256 checksum for the NiFi 1.20.0 distribution. The Flow Definition also includes a required SSL Context Service configured with the Java Runtime cacerts Trust Store.
The Migrated Flow Definition includes InvokeHTTP configured with the same URL but without the SSL Context Service, which is not required for public trusted HTTPS services.
GetJMSQueue
Component Class | org.apache.nifi.processorscontroller.standardscheduling.GetJMSQueueCronSchedulingAgent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standardframework-narcore |
Deprecation TypeComponent | ClassMethod Argument Values |
Deprecated Version | 1.324.0 |
Migration Difficulty | MEDIUMLOW |
Migration Type | Component Replacement |
Alternatives
...
org.apache.nifi.jms.processors.ConsumeJMS
...
Migration
GetJMSQueue supports retrieving messages using the Java Message Service Queue interface to access Apache ActiveMQ.
GetJMSQueue bundles the ActiveMQ libraries and requires the URL property to specify the JMS server location, and the Destination Name property to specify the JMS queue for retrieval. Additional optional properties support TLS communication and authentication as well as batching.
The standard ConsumeJMS Processor similar functionality to GetJMSQueue, but requires additional configuration. ConsumeJMS does not bundle JMS libraries and requires referencing a local directory containing implementation JAR files.
ConsumeJMS should be configured with a Connection Factory Service implementation of JMSConnectionFactoryProvider with required properties. The ActiveMQ Artemis library provides the current supported version of JMS access.
...
Method argument changes |
Migration
Component run scheduling using cron expression changed from using Quartz to Spring Framework CronExpression for parsing the scheduling period in NIFI-12290.
The new parsing approach does not impact most cron expressions, but does introduce the following changes:
- Removed support for the optional Year field as the seventh field in an expression
- Integer definitions for day of the week span from 0 to 7 instead of 1 to 7
Components can no longer be scheduled to run for a particular year, so any referenced to a seventh field in an expression must be removed. Using named day of the week references such as SUN or MON are preferred over integer references.
ExpressionLanguageScope VARIABLE_REGISTRY
In Apache NiFi 2.0, the ExpressionLanguageScope VARIABLE_REGISTRY has been removed. An alternate expression language scope ENVIRONMENT is now available, allowing for similar configurability via JVM system properties or system environment variables. Further detail on this change may be found in NIFI-12079.
Migration
- Logic of existing flow definitions should be updated to no longer depend on the presence of variables at any process group scope, or in the file registry file. Usage of parameter contexts and parameters is recommended to mitigate any gaps in flow definition logic.
- Existing custom NARs should be rebuilt using the NiFi 2.0 API. Any component properties that provide configurability via VARIABLE_REGISTRY should be adjusted to reference one of the current ExpressionLanguageScope options.
Processors
Migrating deprecated Processors and properties requires flow design changes. The steps required range from setting different property values to replacing one Processor with one or more Processors providing similar capabilities.
The section for each deprecation provides the component class name as well as the Deprecation Type and Migration Difficulty. Each section includes Migration steps and example Flow Definitions with selected configuration properties.
Base64EncodeContent
ConsumeJMS uses the Connection Factory Service along with several other properties to access JMS Queues.
...
Flow Definitions
The Deprecated Flow Definition includes GetJMSQueue configured for ActiveMQ running on the localhost address with a specified Destination Name.
The Migrated Flow Definition includes ConsumeJMS configured with a JMSConnectionFactoryProvider using the same JMS broker information. The Migrated Flow Definition includes a JMS Parameters context with a Parameter named JmsClientLibrariesPath referencing the local directory containing the Apache ActiveMQ Artemis libraries.
GetJMSTopic
Component Class | org.apache.nifi.processors.standard.GetJMSTopicBase64EncodeContent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.320.0 |
Migration Difficulty | MEDIUMLOW |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.jmsprocessors.processorsstandard.ConsumeJMSEncodeContent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-jms-processorsstandard-nar |
Migration
GetJMSTopic supports retrieving messages using the Java Message Service Topic interface to access Apache ActiveMQ.
GetJMSTopic bundles the ActiveMQ libraries and requires the URL property to specify the JMS server location, and the Destination Name property to specify the JMS topic for retrieval. Additional optional properties support TLS communication and authentication as well as batching.
The standard ConsumeJMS Processor similar functionality to GetJMSTopic, but requires additional configuration. ConsumeJMS does not bundle JMS libraries and requires referencing a local directory containing implementation JAR files.
ConsumeJMS should be configured with a Connection Factory Service implementation of JMSConnectionFactoryProvider with required properties. The ActiveMQ Artemis library provides the current supported version of JMS access.
...
ConsumeJMS uses the Connection Factory Service along with several other properties to access JMS Queues.
...
Flow Definitions
The Deprecated Flow Definition includes GetJMSTopic configured for ActiveMQ running on the localhost address with a specified Destination Name.
The Migrated Flow Definition includes ConsumeJMS configured with a JMSConnectionFactoryProvider using the same JMS broker information. The Migrated Flow Definition includes a JMS Parameters context with a Parameter named JmsClientLibrariesPath referencing the local directory containing the Apache ActiveMQ Artemis libraries.
GetTwitter
...
org.apache.nifi.processors.twitter.GetTwitter
...
Base64EncodeContent supports a single Mode property that can be set to Encode or Decode.
The EncodeContent Processor supports Base64 and other formats for both encoding and decoding.
EncodeContent provides a direct replacement for Base64EncodeContent using standard properties.
Property | Value |
---|---|
Mode | Encode |
Encoding | base64 |
Flow Definitions
The Deprecated Flow Definition includes Base64EncodeContent configured for encoding.
The Migrated Flow Definition replaces Base64EncodeContent with EncodeContent configured with Base64 encoding.
GetHTTP
Component Class | org.apache.nifi.processors.standard.GetHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.9.0 |
Migration Difficulty | LOW |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.processors.standard.InvokeHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
GetHTTP supports a number of configurable properties to retrieve data using the HTTP GET method. The URL property controls the primary behavior of the Processor.
GetHTTP requires the Filename property to set the corresponding FlowFile attribute for content retrieved. The Processor requires an SSL Context Service when the URL starts with the HTTPS scheme.
The standard InvokeHTTP Processor supports standard HTTP methods including GET. Unlike GetHTTP, InvokeHTTP does not require an SSL Context Service for accessing HTTPS locations as it defaults to the Java Runtime default certificate authorities.
InvokeHTTP has a required HTTP URL property and defaults to setting the FlowFile filename attribute to a random UUID. The filename attribute can be derived from the URL path using the Response FlowFile Naming Strategy property set to URL_PATH.
InvokeHTTP has lower default socket timeout settings than GetHTTP.
Setting standard InvokeHTTP properties provides functionality similar to GetHTTP,
Property | Value |
---|---|
HTTP Method | GET |
HTTP URL | HTTP or HTTPS location |
Response FlowFile Naming Strategy | URL_PATH |
Flow Definitions
The Deprecated Flow Definition includes GetHTTP configured to retrieve the SHA-256 checksum for the NiFi 1.20.0 distribution. The Flow Definition also includes a required SSL Context Service configured with the Java Runtime cacerts Trust Store.
The Migrated Flow Definition includes InvokeHTTP configured with the same URL but without the SSL Context Service, which is not required for public trusted HTTPS services.
GetJMSQueue
Component Class | org.apache.nifi.processors.standard.GetJMSQueue |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.3.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.jms.processors.ConsumeJMS |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-jms-processors-nar |
Migration
GetJMSQueue supports retrieving messages using the Java Message Service Queue interface to access Apache ActiveMQ.
GetJMSQueue bundles the ActiveMQ libraries and requires the URL property to specify the JMS server location, and the Destination Name property to specify the JMS queue for retrieval. Additional optional properties support TLS communication and authentication as well as batching.
The standard ConsumeJMS Processor similar functionality to GetJMSQueue, but requires additional configuration. ConsumeJMS does not bundle JMS libraries and requires referencing a local directory containing implementation JAR files.
ConsumeJMS should be configured with a Connection Factory Service implementation of JMSConnectionFactoryProvider with required properties. The ActiveMQ Artemis library provides the current supported version of JMS access.
Property | Value |
---|---|
JMS Connection Factory Implementation Class | org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory |
JMS Client Libraries | /opt/activemq-artemis-2.28.0/lib |
JMS Broker URI | tcp://${AMQ_HOST}:61616 |
ConsumeJMS uses the Connection Factory Service along with several other properties to access JMS Queues.
Property | Value |
---|---|
Connection Factory Service | JMSConnectionFactoryProvider |
Destination Name | |
Destination Type | QUEUE |
Flow Definitions
The Deprecated Flow Definition includes GetJMSQueue configured for ActiveMQ running on the localhost address with a specified Destination Name.
The Migrated Flow Definition includes ConsumeJMS configured with a JMSConnectionFactoryProvider using the same JMS broker information. The Migrated Flow Definition includes a JMS Parameters context with a Parameter named JmsClientLibrariesPath referencing the local directory containing the Apache ActiveMQ Artemis libraries.
GetJMSTopic
Component Class | org.apache.nifi.processors.standard.GetJMSTopic |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.3.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.jms.processors.ConsumeJMS |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-jms-processors-nar |
Migration
GetJMSTopic supports retrieving messages using the Java Message Service Topic interface to access Apache ActiveMQ.
GetJMSTopic bundles the ActiveMQ libraries and requires the URL property to specify the JMS server location, and the Destination Name property to specify the JMS topic for retrieval. Additional optional properties support TLS communication and authentication as well as batching.
The standard ConsumeJMS Processor similar functionality to GetJMSTopic, but requires additional configuration. ConsumeJMS does not bundle JMS libraries and requires referencing a local directory containing implementation JAR files.
ConsumeJMS should be configured with a Connection Factory Service implementation of JMSConnectionFactoryProvider with required properties. The ActiveMQ Artemis library provides the current supported version of JMS access.
Property | Value |
---|---|
JMS Connection Factory Implementation Class | org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory |
JMS Client Libraries | /opt/activemq-artemis-2.28.0/lib |
JMS Broker URI | tcp://${AMQ_HOST}:61616 |
ConsumeJMS uses the Connection Factory Service along with several other properties to access JMS Queues.
Property | Value |
---|---|
Connection Factory Service | JMSConnectionFactoryProvider |
Destination Name | |
Destination Type | TOPIC |
Flow Definitions
The Deprecated Flow Definition includes GetJMSTopic configured for ActiveMQ running on the localhost address with a specified Destination Name.
The Migrated Flow Definition includes ConsumeJMS configured with a JMSConnectionFactoryProvider using the same JMS broker information. The Migrated Flow Definition includes a JMS Parameters context with a Parameter named JmsClientLibrariesPath referencing the local directory containing the Apache ActiveMQ Artemis libraries.
GetTwitter
Component Class | org.apache.nifi.processors.twitter.GetTwitter |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-social-media-nar |
Deprecation Type | Component Class |
Deprecated Version | 1.17.0 |
Migration Difficulty | MEDIUM |
Migration Type | Component Replacement |
Alternatives
Component Class | org.apache.nifi.processors.twitter.ConsumeTwitter |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-social-media-nar |
Migration
GetTwitter uses Version 1 of the Twitter REST API and relies on the retired Twitter Hosebird Client library.
GetTwitter supports several different endpoints for streaming Tweets, including Sample realtime Tweets and Filter realtime Tweets.
GetTwitter also supports configurable properties for filtering Tweets, but this filtering approach is not support in Version 2 of the Twitter REST API, and is not supported in ConsumeTwitter.
ConsumeTwitter uses Version 2 of the Twitter REST API. Twitter documentation provides a general guide for migrating from Version 1 to Version 2 for filtered streams and sampled streams.
ConsumeTwitter with Twitter API Version 2 requires a single OAuth 2.0 Bearer Token for app-only authentication. This approach replaces the API Key and Access Token approach required for GetTwitter.
Configuring ConsumeTwitter with the Sample Stream setting provides similar behavior to GetTwitter Sample Endpoint setting.
Configuring ConsumeTwitter with the Search Stream setting requires creating Filter Rules using the Twitter REST API outside of processor properties.
Twitter Filter Rules
Twitter Filter Rules can be created using JSON sent as an HTTP POST to the Twitter REST API Version 2.
- https://api.twitter.com/2/tweets/search/stream/rules
The Languages property in GetTwitter can be applied to a Filter Rule using the lang operator in a Filter value definition.
{
"add": [
{
"value": "Apache lang:en",
"tag": "English Language Filter"
}
]
}
The Terms to Filter On property in GetTwitter can be applied to a Filter Rule using the Filter value definition with optional Boolean operators.
{
"add": [
{
"value": "Apache AND NiFi",
"tag": "Boolean Terms Filter"
}
]
}
The IDs to Filter On property in GetTwitter can be applied to a Filter Rule using the from operator in a Filter value definition.
{
"add": [
{
"value": "from:1625695877326340102",
"tag": "Twitter User ID Filter"
}
]
}
Alternatives
...
org.apache.nifi.processors.twitter.ConsumeTwitter
...
Migration
GetTwitter uses Version 1 of the Twitter REST API and relies on the retired Twitter Hosebird Client library.
GetTwitter supports several different endpoints for streaming Tweets, including Sample realtime Tweets and Filter realtime Tweets.
GetTwitter also supports configurable properties for filtering Tweets, but this filtering approach is not support in Version 2 of the Twitter REST API, and is not supported in ConsumeTwitter.
ConsumeTwitter uses Version 2 of the Twitter REST API. Twitter documentation provides a general guide for migrating from Version 1 to Version 2 for filtered streams and sampled streams.
ConsumeTwitter with Twitter API Version 2 requires a single OAuth 2.0 Bearer Token for app-only authentication. This approach replaces the API Key and Access Token approach required for GetTwitter.
Configuring ConsumeTwitter with the Sample Stream setting provides similar behavior to GetTwitter Sample Endpoint setting.
...
Flow Definitions
The Deprecated Flow Definition includes GetTwitter configured with the Sample Endpoint and without filters.
...
Component Class | org.apache.nifi.processors.standard.UpdateAttribute |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
HashAttribute supports generating an MD5 digest of one more FlowFile attribute name and value pairs. The MD5 algorithm is not secure for cryptographic operations, and alternatives such as the SHA-2 or SHA-3 algorithms should be used.
...
Component Class | org.apache.nifi.processors.standard.CryptographicHashContent |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
HashContent supports generating a digest of FlowFile content using a configurable algorithm and writing the hash to a specified FlowFile attribute.
...
Component Class | org.apache.nifi.processors.standard.InvokeHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Deprecation Type | Component Properties |
Deprecated Version | 1.18.0 |
Migration Difficulty | LOW |
Migration Type | Property Replacement |
Migration
InvokeHTTP includes several deprecated properties for configuring access through a proxy server.
...
Component Class | org.apache.nifi.processors.standard.InvokeHTTP |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-standard-nar |
Migration
PostHTTP supports sending FlowFile content to a configurable HTTP URL using the HTTP POST method.
...
Component Class | org.apache.nifi.jms.processors.PublishJMS |
Bundle Group | org.apache.nifi |
Bundle Artifact | nifi-jms-processors-nar |
Migration
PutJMS supports sending messages using the Java Message Service interface to access Apache ActiveMQ.
...