...

Secure ZooKeeper communications

...

In order to implement secure ZooKeeper comunications at least we have to set ACLs which prevent unknown users to modify and/or access ledgers metadata and bookies metadata

...

The idea is to have a global "secure zookeeper comunications", to be set both on bookies and on clients (as bookkeeper clients do the most of the work actually).

Initial proposal is to set the default ZooKeeper ACL ZooDefs.Ids.CREATOR_ALL_ACL to every Z-Node created from BookKeeper clients and Bookies.

This will work if every client and bookie performs authentication to ZooKeeper with the same user/principal. This is very common in the Hadoop ecosystem (see Kafka, HBase...) and with Kerberos bases setups.

Usually is such setups each machine logs in using a principal in the form zkclient/HOSTNAME@REALM and the HOSTNAME and REALM part are stripped out, this way each client gets authenticated as simply "zkclient" user.

Rolling upgrade to switch from authentication types:

My The  proposal is to support the switch from "no-auth" to any plugin is the following:

...

ISSUE: Do we have to add this feature on 4.5.0  as we are breaking compatibility with 4.4 on protocol ?

SASL/Kerberos support:

The initial proposal is to support SASL authentiaction as ZooKeeper does, this is turn is implemented using standard Java support for JAAS and SASL

The admin need to simply write  a JAAS configuration file like this for Kerberos and setup keytabs for each machine.

Bookie {
       com.sun.security.auth.module.Krb5LoginModule required debug=true
       useKeyTab=true
       keyTab=/path/to/server.keytab
       storeKey=true
       useTicketCache=false
       principal=bookkeeper/HOSTNAME@REALM
};

BookKeeper {
       com.sun.security.auth.module.Krb5LoginModule required debug=true
       useKeyTab=true
       keyTab=/path/to/client.keytab
       storeKey=true
       useTicketCache=false
       principal=username/HOSTNAME@REALM
};

For simple DIGEST-MD5 login we can use  the same LoginModule class as ZooKeeper and implement a similar handler

Bookie {       
       org.apache.zookeeper.server.auth.DigestLoginModule required
       user_user1="testpwd";
       user_user2="testpwd";
};
 
BookKeeper {
       org.apache.zookeeper.server.auth.DigestLoginModule required
       username="user1"
       password="testpwd";
};

Following the conventions the "Bookie" principal will be bookeeper/HOSTNAME@REALM (for instance in zookeeper it has to be zookeeper/HOSTNAME@REALM and for kafka it is kafka/HOSTNAME@REALM).

For simple authentication we do not care about dealing with principal manipulations, like stripping out HOSTNAME and REALM, as it it possible in ZooKeeper (see kerberos.removeHostFromPrincipal, kerberos.removeRealmFromPrincipal properties).

In BookKeeper bookie-to-bookie comunication is simply a client-to-bookie comunication and so on Bookie the admin will need to configure both the server-side configuration entry (Bookie) and the client-side configuration entry (BookKeeper).

ISSUE:  On ZookKeeper the SASL mech is decided upon the type of JAAS Subject, this is very simply from admin to be configured. Should we make the configuration more explicit and make the admin configure that the system will run using a specific mech and fail if this mech is not configured at JAAS level ? Something like a configuration property sasl.mech=GSSAPI

 TDB

Action

...