Status
Current state: [Under Discussion]
Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
JIRA:
Released: <Flink Version>
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Aligning the overview page of JobManager with the Flink's new memory model (FLIP-116).
Proposed Changes
This includes making JobManager's memory-related metrics available in the UI. Additionally, the configuration parameters should be exposed similar to the TaskManager's overview (see FLIP-102: Add More Metrics to TaskManager).
JVM | Flink Compose0 | user conf key1 | configuration key2 | metric max3 | metric used3 |
Heap | Framework Heap | memoryConfiguration.frameworkHeap | Status.JVM.Memory.Heap.Max | Status.JVM.Memory.Heap.Used | |
Off-Heap | Framework Off-Heap | jobmanager.memory.off-heap.size | memoryConfiguration.frameworkOffHeap | Status.JVM.Memory.NonHeap.Max | Status.JVM.Memory.NonHeap.Used |
JVM MetaSpace | memoryConfiguration.jvmMetaspace | Status.JVM.Memory.Metaspace.Max | Status.JVM.Memory.Metaspace.Used
| ||
JVM Overhead | memoryConfiguration.jvmOverhead | Status.JVM.Memory.NonHeap.Max - Status.JVM.Memory.Metaspace.Max | Status.JVM.Memory.NonHeap.Used - Status.JVM.Memory.Metaspace.Used | ||
0 The partitioning how it is described in org.apache.flink.runtime.clusterframework.TaskExecutorProcessSpec.
1 These are the configuration parameters used in the Flink configuration.
2 These are the Json paths to address the properties in the HTTP REST API response. Additionally, memoryConfiguration.totalFlinkMemory
and totalProcessMemory
are exposed through the REST API.
3 The metrics which are exposed through the metrics endpoint.
Frontend Design
Display metrics inside JVM, outside JVM and GC
REST API proposal:
Metrics REST Endpoint
The API can be used to retrieve the metrics for the JobManager: http://localhost:8081/jobmanager/metrics?get=Status.JVM.Memory.Heap.Max,Status.JVM.Memory.Heap.Used,Status.JVM.Memory.NonHeap.Max,Status.JVM.Memory.NonHeap.Used,Status.JVM.Memory.Metaspace.Max,Status.JVM.Memory.Metaspace.Used
Test Plan
Covered by unit tests.