Versions Compared

Old Version 14

changes.mady.by.user Ferenc Gerlits

Saved on

compared with

New Version 15

changes.mady.by.user Ferenc Gerlits

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

If you want the data flow to use HTTPS, that is a bit more complicated; below is a description of how to do it.  (Tested with NiFi 12.120.1 0-M1 and MiNiFi c++ 0.715.0.)

Step-by-step guide

...

  1. Create a self-signed certificate

...

  1. ; there are many how-tos on the internet.  Below, we'll assume that your generated files are /opt/certs/agent-cert.pem and /opt/certs/agent-key.pem; the CN of the certificate is "my-agent-ID".

  2. Install NiFi.  When you a access https://<hostname>:9443/nifi/ the first time in your browser, you will get a "Potential Security Risk Ahead" warning about NiFi's self-signed certificate, but you can click Advanced → Accept the Risk and Continue.

  3. Add the following settings to your NiFi installation's nifi.properties file

  4. Download the latest version of nifi-toolkit from https://nifi.apache.org/download.html

  5. Unpack the contents: 

    Code Block
    tar xzvf nifi-toolkit-1.12.1-bin.tar.gz -C /opt/nifi/

    Create a directory for the certificate files:

    Code Block
    mkdir -p /opt/nifi/data/ssl

    Determine the hostname that will be used to access NiFi from web browsers and from MiNiFi. Then generate the certificates using tls-toolkit:

    Code Block
    # SetSite theto location of Java; the executable should be ${JAVA_HOME}/bin/java
export JAVA_HOME=...
# Replace <hostname> with the hostname used to access NiFi
/opt/nifi/nifi-toolkit-1.12.1/bin/tls-toolkit.sh standalone -n "<hostname>" -C "CN=minifi, OU=NIFI" -o /opt/nifi/data/ssl

# Unpack the certificate and key from the .p12 bundle
# Enter the contents of CN=minifi_OU=NIFI.password as the password when prompted (both times)
openssl pkcs12 -in /opt/nifi/data/ssl/CN=minifi_OU=NIFI.p12 -out /opt/nifi/data/ssl/nifi-rest.key -nocerts -nodes
openssl pkcs12 -in /opt/nifi/data/ssl/CN=minifi_OU=NIFI.p12 -out /opt/nifi/data/ssl/nifi-rest.crt -clcerts -nokeys

    Protect the files by ensuring that only the current user have access to them:

    Code Block
    chmod 755 /opt/nifi/data/ssl
chmod 600 /opt/nifi/data/ssl/CN\=minifi_OU\=NIFI.*
chmod 700 /opt/nifi/data/ssl/<hostname>

    Copy the value of the properties in the nifi.remote, nifi.web, nifi.security sections from /opt/nifi/data/ssl/<hostname>/nifi.properties to your NiFi installation's nifi.properties file, except the for the following two, which should be set like this:

    Code Block
    nifi.security.keystore=/opt/nifi/data/ssl/<hostname>/keystore.jks
nifi.security.truststore=/opt/nifi/data/ssl/<hostname>/truststore.jks

    and add the following settings to your NiFi installation's nifi.properties file:

    Code Block
    nifi.rest.host=<hostname>
nifi.rest.keystorePath=/opt/nifi/data/ssl/CN=minifi_OU=NIFI.p12
nifi.rest.keystorePassword=/opt/nifi/data/ssl/CN=minifi_OU=NIFI.password
nifi.rest.keystoreType=PKCS12
    Site properties
nifi.remote.input.host=localhost
nifi.remote.input.secure=true
nifi.remote.input.socket.port=7777
nifi.remote.input.http.enabled=true


  6. Export the NiFi certificate from the NiFi trust store, and import the MiNiFi agent certificate into the NiFi trust store: 

    Code Block
    keytool -exportcert -alias nifi-cert -rfc -keystore truststore.p12 > /opt/certs/nifi-cert.pem
keytool -importcert -alias agent-cert -file /opt/certs/agent-cert.pem -keystore truststore.p12

    keytool will prompt you for the truststore password; you can get this from the nifi.properties file.

  7. Add the agent ID (which is the CN of the agent certificate) as the "Initial User Identity 1" value in the "userGroupProvider" Uncomment the file-provider section of authorizers.xml in your NiFi installation, and set the Initial Admin Identity:

    Code Block
        <authorizer><userGroupProvider>
        <identifier>file-user-group-provider</identifier>
        <class>org.apache.nifi.authorization.FileAuthorizer<FileUserGroupProvider</class>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Users File">./conf/users.xml</property>
        <property name="Initial AdminUser Identity">CN=minifi, OU=NIFI</property>
        <property name="Legacy Authorized Users File"></property>

        <!-- Provide the identity (typically a DN) of each node when clustered, see above description of Node Identity. 1">my-agent-ID</property>
    </userGroupProvider>

    and restart NiFi; it will create a new user in users.xml which looks like this:

    Code Block
    <tenants>
    <groups/>
    <users>
        <property<user nameidentifier="Node Identity 1"></property>
        --9a889e09-6e86-360a-a324-8f3ee341842a" identity="my-agent-ID"/>
    </authorizer>

    and restart NiFi; it will create a new user with your Initial Admin Identity name in users.xml .

    users>
</tenants>


  8. Add authorizations for your user in authorizations.xml in your NiFi installation, copying the user identifier from users.xml:

    Code Block
    <authorizations>
  <!-- generate UUIDs for the policy identifiers -->
  <policies>
        <policy identifier="1f6ae57a-08bc-11eb-9242-bf69163fde10" resource="/site-to-site" action="R">
          <!-- copy the user identifier from users.xml -->
          <user identifier="9cd6ec429a889e09-b3da6e86-3a3a360a-8405a324-7264746e0e428f3ee341842a"/>
        </policy>
        <policy identifier="282818e0-08bc-11eb-8508-2b51c9d70d42" resource="/site-to-site" action="W">
          <user identifier="9cd6ec429a889e09-b3da6e86-3a3a360a-8405a324-7264746e0e428f3ee341842a"/>
        </policy>
         
        <!-- copy the port identifier from Remote Process Groups/Input Ports/id in the minifi config.yml -->
        <policy identifier="f512f796-7afb-4c9f-ab68-b5eaf6d5d0cf" resource="/data-transfer/input-ports/c171f9da1342ea64-689f018d-41e21000-98c4a5cf-9d785c59c3067bda6830909b" action="R">
          <user identifier="9cd6ec429a889e09-b3da6e86-3a3a360a-8405a324-7264746e0e428f3ee341842a"/>
        </policy>
        <policy identifier="b4e836ee-d526-4e16-8bf3-ee1d8fa3d5e6" resource="/data-transfer/input-ports/c171f9da1342ea64-689f018d-41e21000-98c4a5cf-9d785c59c3067bda6830909b" action="W">
          <user identifier="9cd6ec429a889e09-b3da6e86-3a3a360a-8405a324-7264746e0e428f3ee341842a"/>
        </policy>
  
        <!-- you will also need a pair of policies for resource="/data-transfer/output-ports/..." if you want S2S data transfer from NiFi to MiNiFi -->
   </policies>
</authorizations>


Then restart NiFi again, and test that you can connect to it using https:

Code Block
# This should give a large HTML response
curl -k --key /opt/nifi/data/ssl/nifi-rest.keycerts/agent-key.pem --cert /opt/nifi/data/ssl/nifi-rest.crtcerts/agent-cert.pem https://<hostname>:94438443/nifi/

# This should give a JSON response
curl -k --key /opt/nifi/data/ssl/nifi-rest.keycerts/agent-key.pem --cert /opt/nifi/data/ssl/nifi-rest.crtcerts/agent-cert.pem https://<hostname>:94438443/nifi-api/site-to-site/

For Mac, https://kylo.readthedocs.io/en/v0.10.0/security/ConfigureNiFiWithSSL.html has instructions on how to tell your browser to trust this self-signed certificate.  On Firefox/Windows, follow these steps: https://support.globalsign.com/digital-certificates/digital-certificate-installation/install-client-digital-certificate-firefox-windows.  On Firefox/Linux, the steps are similar; the View Certificates dialog is at Preferences → Privacy & Security (scroll to the bottom).  After you have added the CN=minifi_OU=NIFI.p12 client certificate under Your Certificates (enter the contents of CN=minifi_OU=NIFI.password when prompted), and you try to access https://<hostname>:9443/nifi/, you will still get a "Potential Security Risk Ahead" warning about the certificate, but now you can click Advanced → Accept the Risk and Continue (you only need to do this the first time).


If everything is OK so far, then configure MiNiFi to use the certificatesecure site-to-site:

  1. Set up the certificate certificates in minifi.properties:

    Code Block
    nifi.remote.input.secure=true
nifi.security.need.ClientAuth=true
nifi.security.client.certificate=/opt/nifi/data/ssl/nifi-rest.crtcerts/agent-cert.pem
nifi.security.client.private.key=/opt/nifi/data/ssl/nifi-rest.keycerts/agent-key.pem
nifi.security.client.pass.phrase=<key passphrase if any>
nifi.security.client.ca.certificate=/opt/nifi/datacerts/ssl/nifi-cert.pem

# These are not needed
#nifi.rest.api.user.name=
#nifi.rest.api.password=


  2. Update the NiFi address in config.yml by changing http  to https and changing the port from 8080 to 9443 8443 in the Remote Process Group:

    Code Block
    Remote Process Groups:
- id: 1ca9d943-0175-1000-2188-4d25f7418459
  name: https://<hostname>:94438443/nifi/
  url: https://<hostname>:94438443/nifi/
  comment: ''
  timeout: 30 secs
  yield period: 10 sec
  transport protocol: RAW
  proxy host: ''
  proxy port: ''
  proxy user: ''
  proxy password: ''
  local network interface: ''
  Input Ports:
  - id: c171f9da-689f-41e2-98c4-9d785c59c306
    name: c171f9da-689f-41e2-98c4-9d785c59c306
    comment: ''
    max concurrent tasks: 1
    use compression: true
  Output Ports: []

    And restart MiNiFi.  Note that the site-to-site port you configured in NiFi will be used for communication, but in the MiNiFi config.yml you need to specify the main NiFi port 8443, NOT the site-to-site port (7777 in the example above).

Content by Label
showLabelsfalse
max5
spacesMINIFI
showSpacefalse
sortmodified
reversetrue
typepage
cqllabel in ("sitetosite","https","tls") and type = "page" and space = "MINIFI"
labelsHTTPS TLS SiteToSite

...