Page History

Recent commits have allowed us to create extensions with code and dependencies that can be included conditionally. This will allow MiNiFi C++ consumers to select the features that they desire, help us build a cohesive API within libminifi, and support patches/upgrades with shared objects. The next first step is to move functional components into this paradigm.

The first component to move will be processors. There has always been a small selection of base processors within MiNiFi. Moving these to a separate extension will allow consumers to include processors conditionally. The design of this extension will be specialized to allow processors/or controller services to be selected in groups or individually. Further, these selected components will be built within the executable and as a shared object that can be versioned as a deliverable to MiNiFi agents. This will be integral to the update framework that will be discussed shortly.

...

This approach will create a binary with only the GetFile processor and RemoteProcessGroup along with a shared object that can be used for patching or updates.  The addition of new base processors will not require the addition of any cmake CMAKE code or code beyond the processor and test. The shared object will include a version that identifies the built source and enables agents that load that shared object to enforce update rules. While a binary limited to GetFile and an RPG may not be useful to many, including updates to either can allow us to update agents in real time through the transmission of a shared object. Realtime updates with non updates to system dependencies can be done in realtime. This can be done by versioning the class loading mechanism. While we don't aim to match the complexity of java class loading, we can certainly use the facilities that exist now to separate class instantiation. Once registry versioning is complete, we should be able to support object isolation.

...

• The major negative to this has been one we've already experienced: increased cost in vtables lookup and class loader lookup 1. While we can't combat the former, we have done some work with template meta programming to fight the latter. More can be done; however, the cost will always exist, but is traditionally very minimal and paid at startup. Nonetheless, we already pay these costs so it will be difficult to argue that we pay the cost more. 
• Design complexity may make development more difficult.
• Debugging may become more difficult due to the depth of call graphs. 

...