(Tested with NiFi 2.0.0-M1 and MiNiFi c++ 0.15.0.)
- Install NiFi. When you a access https://<hostname>:8443/nifi/ the first time in your browser, you will get a "Potential Security Risk Ahead" warning about NiFi's self-signed certificate, but you can click Advanced → Accept the Risk and Continue.
Create a NiFi flow with an Input Port. Double-click the Input Port, and note its ID (1342ea64-018d-1000-a5cf-7bda6830909b in the screenshot.)
- Create a MiNiFi flow with a Remote Processing Group and an Input Port as shown in this example: https://github.com/apache/nifi-minifi-cpp/blob/main/examples/site_to_site_config.yml. Update
Remote Processing Groups/Input Ports/id
andConnections[0]/id
to the Input Port ID from NiFi, andRemote Processing Groups/Input Ports/url
to the NiFi URL you use to access NiFi in your browser.
- Create a self-signed certificate; there are many how-tos on the internet. Below, we'll assume that your generated files are /opt/certs/agent-cert.pem and /opt/certs/agent-key.pem; the CN of the certificate is "my-agent-ID".
Add the following settings to your NiFi installation's
nifi.properties
file:Code Block # Site to Site properties nifi.remote.input.host=localhost nifi.remote.input.secure=true nifi.remote.input.socket.port=7777 nifi.remote.input.http.enabled=true
Note that the port you configure here, 7777 in this example, will be used internally by the site-to-site communication, but in the MiNiFi
config.yml
file, you should use the same NiFi address you use in your browser, NOT this site-to-site port.Export the NiFi certificate from the NiFi trust store, and import the MiNiFi agent certificate into the NiFi trust store:
Code Block keytool -exportcert -alias nifi-cert -rfc -keystore truststore.p12 > /opt/certs/nifi-cert.pem keytool -importcert -alias agent-cert -file /opt/certs/agent-cert.pem -keystore truststore.p12
keytool will prompt you for the truststore password; you can get this from the
nifi.properties
file.Add the agent ID (which is the CN of the agent certificate) as the "Initial User Identity 1" value in the "userGroupProvider"
It is fairly easy to set up an unencrypted data flow from MiNiFi to NiFi: https://nifi.apache.org/minifi/getting-started.html.
If you want the data flow to use HTTPS, that is a bit more complicated; below is a description of how to do it. (TODO: this doesn't quite work, yet, complete it)
Step-by-step guide
First, go to https://kylo.readthedocs.io/en/v0.10.0/security/ConfigureNiFiWithSSL.html and create a self-signed certificate as described there:
- Download the latest version of
nifi-toolkit
from https://nifi.apache.org/download.html Unpack the contents:
Code Block tar xzvf nifi-toolkit-1.12.1-bin.tar.gz -C /opt/nifi/
Create a directory for the certificate files:
Code Block mkdir -p /opt/nifi/data/ssl
Determine the hostname(s) that will be used to access NiFi from web browsers and from MiNiFi. Then generate the certificates using tls-toolkit:
Code Block # Set the location of Java; the executable should be ${JAVA_HOME}/bin/java export JAVA_HOME=... # Replace <hostname> with the comma-separate list of hostnames used to access NiFi /opt/nifi/nifi-toolkit-1.12.1/bin/tls-toolkit.sh standalone -n '<hostname>' -C 'CN=minifi, OU=NIFI' -o /opt/nifi/data/ssl # Unpack the certificate and key from the .p12 bundle # Enter the contents of CN=minifi_OU=NIFI.password as the password when prompted (both times) openssl pkcs12 -in /opt/nifi/data/ssl/CN=minifi_OU=NIFI.p12 -out /opt/nifi/data/ssl/nifi-rest.key -nocerts -nodes openssl pkcs12 -in /opt/nifi/data/ssl/CN=minifi_OU=NIFI.p12 -out /opt/nifi/data/ssl/nifi-rest.crt -clcerts -nokeys
Protect the files by ensuring that only the current user have access to them:
Code Block chmod 755 /opt/nifi/data/ssl chmod 600 /opt/nifi/data/ssl/CN\=minifi_OU\=NIFI.* chmod 700 /opt/nifi/data/ssl/<hostname>
Copy the value of the properties in the
nifi.remote
,nifi.web
,nifi.security
sections from/opt/nifi/data/ssl/<hostname>/nifi.properties
to your NiFi installation'snifi.properties
file, except the for the following two, which should be set like this:Code Block nifi.security.keystore=/opt/nifi/data/ssl/<hostname>/keystore.jks nifi.security.truststore=/opt/nifi/data/ssl/<hostname>/truststore.jks
and add the following settings to your NiFi installation's
nifi.properties
file:Code Block # Replace <keystore-password> with the contents of /opt/nifi/data/ssl/CN=minifi_OU=NIFI.password nifi.rest.host=<hostname> nifi.rest.keystorePath=/opt/nifi/data/ssl/CN=minifi_OU=NIFI.p12 nifi.rest.keystorePassword=<keystore-password> nifi.rest.keystoreType=PKCS12
Uncomment the
file-provider
section ofauthorizers.xml
in your NiFi installation, and set the Initial Admin Identity:Code Block <authorizer><userGroupProvider> <identifier>file-user-group-provider</identifier> <class>org.apache.nifi.authorization.FileAuthorizer<FileUserGroupProvider</class> <property name="AuthorizationsUsers File">./conf/authorizationsusers.xml</property> <property name="Users File">./conf/users.xml</property> Initial User Identity 1">my-agent-ID</property> </userGroupProvider>
and restart NiFi; it will create a new user in
users.xml
which looks like this:Code Block <tenants> <groups/> <users> <property name="Initial Admin Identity">CN=minifi, OU=NIFI</property><user identifier="9a889e09-6e86-360a-a324-8f3ee341842a" identity="my-agent-ID"/> </users> </tenants>
Add authorizations for your user in
authorizations.xml
in your NiFi installation, copying the user identifier fromusers.xml
:Code Block <authorizations> <policies> <property<policy nameidentifier="Legacy Authorized Users File"></property> 1f6ae57a-08bc-11eb-9242-bf69163fde10" resource="/site-to-site" action="R"> <!-- Providecopy the identity (typically a DN) of each node when clustered, see above description of Node Identity.user identifier from users.xml --> <user identifier="9a889e09-6e86-360a-a324-8f3ee341842a"/> </policy> <property<policy nameidentifier="Node Identity 1"></property>282818e0-08bc-11eb-8508-2b51c9d70d42" resource="/site-to-site" action="W"> <user identifier="9a889e09-6e86-360a-a324-8f3ee341842a"/> </authorizer>
and restart NiFi; it will create a new user with your Initial Admin Identity name in
users.xml
.Add authorization to your user to access the
/site-to-site
API inauthorizations.xml
in your NiFi installation:Code Block <!-- generate some UUIDs for the policy identifiers --> policy> <!-- copy the userport identifier from users.xml the Input Port in NiFi into the resource string --> <policy identifier="1f6ae57af512f796-08bc7afb-11eb4c9f-9242ab68-bf69163fde10b5eaf6d5d0cf" resource="/site-to-site/data-transfer/input-ports/c171f9da-689f-41e2-98c4-9d785c59c306" action="R"> <user identifier="9cd6ec429a889e09-b3da6e86-3a3a360a-8405a324-7264746e0e428f3ee341842a"/> </policy> <policy identifier="282818e0b4e836ee-08bcd526-11eb4e16-85088bf3-2b51c9d70d42ee1d8fa3d5e6" resource="/site-to-site/data-transfer/input-ports/c171f9da-689f-41e2-98c4-9d785c59c306" action="W"> <user identifier="9cd6ec429a889e09-b3da6e86-3a3a360a-8405a324-7264746e0e428f3ee341842a"/> </policy>
TODO: I think more authorizations are needed.
For Mac, https://kylo.readthedocs.io/en/v0.10.0/security/ConfigureNiFiWithSSL.html has instructions on how to tell your browser to trust this self-signed certificate. TODO: add instructions for Linux and Windows, as well.
<!-- you will also need a pair of policies for resource="/data-transfer/output-ports/..." if you want S2S data transfer from NiFi to MiNiFi --> </policies> </authorizations>
Restart
...
NiFi again, and test that you can connect to it using https:
Code Block # This should give a large HTML response curl -k --key /opt/
...
certs/agent-key.pem --cert /opt/
...
certs/agent-cert.pem https://<hostname>:
...
8443/nifi/ # This should give a JSON response curl -k --key /opt/
...
certs/agent-key.pem --cert /opt/
...
certs/agent-cert.pem https://<hostname>:
...
8443/nifi-api/site-to-site/
...
Finally, configure MiNiFi to use
...
hidden | true |
---|
...
secure site-to-site:
Set up the certificate in
minifi.properties
:Code Block nifi.remote.input.secure=true nifi.security.need.ClientAuth=falsetrue nifi.security.client.certificate=/opt/nifi/data/ssl/nifi-rest.crtcerts/agent-cert.pem nifi.security.client.private.key=/opt/nifi/data/ssl/nifi-rest.keycerts/agent-key.pem nifi.security.client.pass.phrase=/opt/nifi/data/ssl/CN=minifi_OU=NIFI.password<key passphrase if any> nifi.security.client.ca.certificate=/opt/nificerts/data/ssl/nifi-cert.pem # These are not needed #nifi.rest.api.user.name= #nifi.rest.api.password=
Update the port in
config.yml
from 8080 to 9443 in the Remote Process Group:Code Block ... Remote Process Groups: - id: c171f9da-689f-41e2-98c4-9d785c59c306 name: http://<hostname>:9443/nifi/ url: http://<hostname>:9443/nifi/ comment: '' ...
And restart MiNiFi.
Related articles
Content by Label | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
- Restart MiNiFi. It should be sending data to NiFi now. Congratulations!