Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Many of these ideas stem from the initial MiNiFi DISCUSS thread on 9 January 2016.  

Page properties
Target release
Document status
Status
titleDRAFT
Document owner

Aldrin Piri

Designer
Developers

...

  • Design and Deploy mechanisms.  How new flows/processing are introduced may vary

Requirements, Feature Requests, Design Considerations

#TitleUser StoryJIRA Issue(s)ImportanceNotes
1Centralized managementA user will want to have a comprehensive view of their assets and handle both command and control of the information they are collecting, transforming, and transmitting. This level of management goes beyond simple deployment but also deal with the alteration and management of the running dataflow. 
  • Data
  • Statistics of Performance/Availability
2Support for/implementation in native formatsThe JVM itself has a large footprint that can be prohibitive in some environments. While the JVM provides a great way to afford reuse of components developed and tested with considerable resources and establish necessary interface and architecture considerations with a known quantity. Other implementations should be possible through well defined interfaces, specifications, and libraries as possible.   
3Vetting/Verification of Provenance / Chain of Custody / Lineage information - Integrity & Non-repudiationDue to the potential difference in trust model, there are other considerations for how we can both trust and potentially verify information provided by endpoint agents.  
4Docker Logging DriversProvide implementations that allow the collection and management of container information   
5Support Agent ExtensibilityIncorporation of a scripting engine or similar facility can allow individuals to cater to local needs without the need for recompilation of code.  
  • Heka: Supports extensions in Go or Lua
  • Hindsight:
6Security ConsiderationsThere needs to be explicit security in both the control and data planes   
7Support common protocols and systemsAs NiFi is flexible and adaptable to various formats, schemas, and protocols, similar provisions would be useful for users of the agent. Scoped to the types of data this model is a natural fit for brings certain best in class protocols to mind. Exemplary protocols provided in Notes.  
  • MQTT
  • AMQP
  • STOMP
8Back-pressure over network boundariesMuch as there is backpressure within connections, it could be advantageous to apply this concept across network boundaries.  Reactive Socket:  http://reactivesocket.io

User interaction and design

...