...
ID | IEP-35 |
Author | |
Sponsor | Nikolay Izhikov
|
Created | |
Status | colourGrey | title | DRAFT - Phase 1,2 implemented. |
Motivation
For now, Ignite has not full, fragmented monitoring API. Those APIs uses use different protocols, such as - JMX, Java API, SQL System views, text logs, etc.
...
1. We should add some entities in Ignite:
- MetricDomain MetricRegistry - Ignite subsystem that provides some set of sensors and lists.
- Cache,
- Compute,
- ServiceGrid,
- etc.
Sensor Metric - some named number with a well-defined algorithm to calculate the value in any given moments in time.
Code Block |
---|
|
class SensorMetric {
String name; //EntryCount, MemoryAvailable, etc
long value; //or double
Collection<Tuple2<String, String>> labels; //hostName, cacheName, etc.
}
class TimeSensorLongMetric extends SensorMetric {
long ts; //timestamp of the last value update.
} |
- List SystemView - some named list of string that contains info about Ignite objects. Examples: List of caches, Transaction Transactions list, List of nodes, List of running queries, Las N queries, etc...
MonitoringEvent - generated when some user-defined code violates the threshold.
Code Block |
---|
|
class MonitoringEvent {
MonitoringEventType type; //Event type.
T info; //Event info. Type of info differs for different type of events.
} |
2. SensorProcessorGridMetricManager, MonitoringEventProcessorGridSystemViewManager:
- SensorProcessor GridMetricManager - should be able to store and query Ignite sensorsmetrics.
- MonitoringEventProcessor GridSystemViewManager - should be able to set up event listeners, watch for user code executions and route eventsstore and export SystemViews.
3. ExposersExporters:
Specific admin interfaces will be supported through exposersexporters.
Exposer Exporters should work only with SensorProcessor only with a read-only version of GridMetricManager and don't rely on other knowledge about Ignite internals.
...
Example of exporters:
- JMX
- HTTP
- SQL System View
- JavaLog
- etc.
- PushExposer - this type of exposers should export sensors and list to some external system based on the configured schedule.
- LogExposer
- Integration with proprietary monitoring system can be implemented as PushExposer.
...
Lists of Ignite objects/entities that should be listed in Phase
...
2
- A list of compute Compute tasks:
- Closures
- Map-reduce jobs
- ComputeJob
- Scheduled tasks
- Service grid:
- Services A list of services with deployment status
- Caches
- Cache groups
- Cluster nodes
- SQL objects
- Schemas
- Tables
- Views
- Tables columns
- Views columns
- Indexes
- Queries:
- SQL
- Scan
- Text
- ContinousQuery
- IgniteCache#invoke
- put, get, remove, replace, clear operations
- Transactions with lock list
- DataStreamers
- Explicit locks(IgniteCache#lock)
- DataStructures
- Queue
- Set
- AtomicLong
- AtomicReference
- CountDownLatch
- Sequence
- Semaphore
- Message topics (IgniteMessaging)
- Thin client connections.
- Machine Learning - ???
...
We should consider implementing this IEP as Ignite 3.
Discussion Links
http://apache-ignite-developers.2346864.n4.nabble.com/IEP-35-Monitoring-amp-Profiling-Proof-of-concept-td41904.html
http://apache-ignite-developers.2346864.n4.nabble.com/IEP-35-Monitoring-amp-Profiling-Current-API-Analysis-td41823.html
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-IEP-35-Metrics-configuration-td42478.html
http://apache-ignite-developers.2346864.n4.nabble.com/IEP-35-GridJobProcessorMetrics-migration-td42415.html#a42441
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-IEP-35-Replace-RunningQueryManager-with-GridSystemViewManager-td43794.html// Links to discussions on the devlist, if applicable.
Gap analysis
Current monitoring APIs availability:
...