Revload497

This is a hands-on walkthrough configuring SSL/TLS authentication in Apache NiFi.The tasks we will accomplish include:

To do so, open the Preferences window (in Edit menu), and click on the Advanced tab. You can go in Encryption tab and click on “View Certificates” button. In “Your Certificates” tab, you can click on the Import button and choose the client.pfx keystore file. The client uses the imported certificate to trust the server that owns that certificate. When a client establishes a session, the server sends a server certificate to the client. If the certificate is a member of the certificates included in the client keystore, the client trusts the server and so proceeds to the session. Learn more about client-side SSL certificates and using the Apache HTTP Client 4.3 in this quick tutorial. We basically create a custom SSL Context, load the keystore. Prepare the Certificate Keystore: Tomcat currently operates only on JKS, PKCS11 or PKCS12 format keystores. The JKS format is Java's standard 'Java KeyStore' format, and is the format created by the keytool command-line utility. This tool is included in the JDK. The PKCS12 format is an internet standard, and can be manipulated via (among other things) OpenSSL and Microsoft's Key-Manager.

1. Creating and installing a user certificate
2. Setting up the server's KeyStore
3. Setting up the server's TrustStore
4. Installing the user certificate into the TrustStore
5. Configuring authorization for our user

At the end, our user will be able to securely log in to our NiFi server.Our walkthrough will use self-signed certificates.The good news is that this simplifies many tasks and gets to a running secure installation very quickly.The bad news is that this may not match your experience working with real, signed certificates in some respects.Getting a firm grip on the process and the end result is critical, more so than the details of working with a particular certificate authority.

For an overview of NiFi security, please take a look at Overview of X.509 SSL Security on NiFi.For a streamlined approach to configuring security in NiFi quickly and easily, see Introducing NiFi-Init.

User Certificate

We're going to start with the admin user certificate. The first step here is to create a private key and public key certificate pair.For example purposes, we will do this for an imaginary 'Admin Q. User'.

You should see output like the following from OpenSSL:

If you are doing this on Windows and get an error 'Subject does not start with '/', please see the Troubleshooting notes below.

In the above openssl command, we create a new 2048-bit RSA key pair consisting of both a private key and certificate with public key.The certificate is valid for 365 days.The -subj argument specifies the certificate identity or 'Distinguished Name' (DN).For now, I've just used the CN (common name), C (country), and L (locality).We can put anything we want into this test certificate, but there are standards and Certificate Authority rules that specify standard organizational, geographical, and personal data fields.Just remember that we will need this Distinguished Name later to configure NiFi authorization for this user.Last, we used the -nodes parameter to not specify a password for the key.OpenSSL writes out two files for both the private key and the public key certificate, both in PEM encoding.

Packaging the User Certificate

Next, we need to package the key pair into an archive acceptable to your web browser.This can be a point of confusion because browsers prompt you for certificate files by file extensions like .pem, .cer, or .pfx, without clarifying what the actual storage format should be.I recommend he PKCS-12 format because it is widely acceptable to browsers, typically saved as a .pfx file.Our PKCS-12 file is protected with a password to prevent casual misuse.

Attaching the Certificate to a Browser

Last, we will import the PFX file into our browser or OS store for a user's personal keys.Pretty much every web browser has a utility for managing certificates found via 'Settings', 'Advanced Settings', 'Security'.In Google Chrome, for example, follow these steps:

1. Select Settings from the menu
2. Scroll down to the bottom and click Show advanced settings...
3. Find the HTTPS/SSL heading and click the button for Manage certificates...
4. On the Personal tab, click the Import... button
5. Go through the wizard to import your .PFX file.You will be prompted to enter the passphrase we created the .PFX file with earlier.Import the certificate to the 'Personal' store.

Our test admin user is now ready to log into a NiFi server, which we will now configure.

Configuring the Server

Our NiFi server needs three things

1. KeyStore
2. TrustStore
3. Authorization settings

Creating the KeyStore

The server's KeyStore is an archive containing the private key and public key certificate identifying the server.We can create a KeyStore file with a key pair in one go using the JDK's Keytool utility.

This creates the KeyStore file in Java KeyStore format (.jks).Both our KeyStore and our key are protected by our super-secret password.As we did above with the admin user, we have taken gross liberties with the Distinguished Name (-dname argument).Typically, the Common Name portion of the Distinguished Name should match the public DNS name of the server.Since we are just testing NiFi configuration, this is not important.

Creating the TrustStore

The TrustStore contains public key certificates for users trusted to log in to this NiFi server in any way.Detailed role authorization will be configured separately below.

Ssl Truststore

First, upload or copy the admin users's public key certificate file, admin-cert.pem to your server.

Next, we'll use the keytool utility again to create a TrustStore archive with this certificate.

And we get a bit of output:

Authorization

NiFi's authorization data matches standard roles with specific users.This information is stored in conf/authorized-users.xml.The file comes pre-populated with helpful comments, we'll fill in the template with edits for our admin user:

Our admin user is identified by Distinguished Name, the same data we gave to OpenSSL's -subj parameter above, but reformated as comma-delimited pairs.The ROLE_ADMIN permissions are specific to editing permissions for users, rather than an all-powerful-super-user flag.So we'll also give our admin ROLE_DFM to allow manipulation of the data flow.

Configuration for the KeyStore and TrustStore

Now we need to fill in the appropriate sections of our nifi.properties configuration file for this KeyStore:

Also, we need to edit the web configuration to reflect HTTPS, by commenting out HTTP port 8080 and configuring HTTPS on port 8443 (typically):

Testing it Out

That should do it, let's test it out.After editing the security settings, we should watch our logs/nifi-app.log carefully to make sure NiFi is happy with our settings.

When the dust settles, we can see that the server has started successfully and is waiting for clients:

One thing to note in the output above is that the URLs should reflect the HTTPS protocol and the port we configured above.If there is a firewall between you and your server, you need to make sure port 8443 is open.I had to update my EC2 Security Group to allow 8443 instead of 8080.

When you open NiFi in your browser at https://yournifiserver:8443/nifi, you will be prompted with the following scary-looking message:

In this case, this nasty-gram does not mean your credit card has been stolen, you are just using a self-signed certificate that your browser does not trust.We can accept this shortcoming and proceed.

And... we're in! We are now securely logged into NiFi as our admin user.You can see the Admin Q. User common name in the top-right toolbar.

Wrapping Up

I hope you found it helpful to take a hands-on walkthrough of NiFi security configuration.Of course, our test setup leaves many things to be desired

• Trust - proper TLS/SSL security involves a trust chain, where clients and servers have some mutual basis for trusting each other,and users are not trained to robotically click through those dire security warning dialogs.
• Key and Certificate Formats - There are a baffling array of certificate file formats, extensions, and encodings.It's pretty easy to generate a self-signed cert, things get much more interesting when trust chains are involved.
• Cloud Deployment - This type of manual configuration is not a great fit for a short-lived cloud server.

We'll come back to these issues in future posts.

Soapui Ssl Keystore

Further Reading

Tomcat Ssl Keystore

Some other resources to check out:

• Hortonworks: Securing NiFi Step-by-Step by Brad Bukacek Jr -This is a concise walkthrough of the steps to secure NiFi, using the TinyCert test certificate generation tool.Using TinyCert results in more realistic test certificate files and allows you to experiment with establishing trust.

Troubleshooting Notes

OpenSSL errors in Git Bash on Windows

Using MinGW64 or Git Bash on Windows, the -subj parameter to OpenSSL should be modified according to this StackOverflow answer.The modification works around the built-in Windows->Unix path conversion logic.OpenSSL receives a converted subject and that results in the 'Subject does not start with '/' error.

Distinguished Name Ordering

While testing out my own instructions I ran into an unexpected issue on my first test of the completed setup.I was prompted by the 'Submit Justification' dialog below:

What Is Keystore

I was just about to write a 500-character justification for my existence, but I eventually figured out this was not a philosophical question,but a request for access form.I believe this was caused by a mismatch between the client Distinguished Name read by the server from the certificate vs.the way I had transcribed it into the authorized_users.xml file.I changed the order of the fields in authorized_users.xml to match the error message that followed this dialog, and it worked.This needs more experimentation and perhaps a separate post, I'm not confident in my understanding here.