THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
According to FLIP-49, we can sort out a table containing the correspondence between configuration and metric.
JVM Metrics
These JVM metrics are exposed and can be used through the TaskManager's metrics REST API.
| JVM | Metric | Used key | Total key |
|---|---|---|---|
| Heap | Status.JVM.Memory.Heap | Used | Max |
| Direct |
Flink Compose0
user conf key1
configuration key2
metric max3
metric used3
Heap
FrameWork Heap
| Status.JVM.Memory. |
| Direct | Used | Max |
| Metaspace | Status.JVM.Memory. |
Task Heap
memoryConfiguration.taskHeap
Off-Heap
Managed Memory
memoryConfiguration.managedMemory
Status.ManagedMemory.Total Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-14406
Status.ManagedMemory.Used Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-14406
Metaspace
| Used | Max | ||||||||
| Mapped | Status.JVM.Memory.Mapped | MemoryUsed | TotalCapacity | |||||||
| NonHeap | Status.JVM.Memory.NonHeap | MemoryUsed | TotalCapacity |
Memory Configuration
Flink's memory model (as described in org.apache.flink.runtime.clusterframework.TaskExecutorProcessSpec) can be mapped to the following Flink configuration parameters. There are a few that have a correlating Flink metric.
| Flink Memory Model | Flink configuration1 | REST API2 | Metric3 | Used key | Total key |
|---|---|---|---|---|---|
| Framework Heap | taskmanager.memory.framework.heap.size | memoryConfiguration.frameworkHeap | Status.JVM.Memory.Heap | Used | Max |
| Task Heap | taskmanager.memory.task.heap.size | memoryConfiguration.taskHeap | |||
| Framework OffHeap | taskmanager.memory.framework.off-heap.size | memoryConfiguration.frameworkOffHeap |
Framework OffHeap
memoryConfiguration.frameworkOffHeap
-
-
Task OffHeap Memory
memoryConfiguration.taskOffHeap
Network Memory
memoryConfiguration.networkMemory
Status.Shuffle.Netty.TotalMemory Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-14422
Status.Shuffle.Netty.UsedMemory Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-14422
| - | - | - |
Status.JVM.Memory.Mapped.TotalCapacity
Status.JVM.Memory.Mapped.MemoryUsed
| Task OffHeap | taskmanager.memory.task.off-heap.size | memoryConfiguration.taskOffHeap |
| Network Memory |
| memoryConfiguration. |
| networkMemory | Status. |
Shuffle.Netty |
|
| UsedMemory | TotalMemory | ||
| Managed Memory | taskmanager.memory.managed.size | memoryConfiguration.managedMemory | Status.ManagedMemory |
|
| (is up for discussion, still) | (is up for discussion, still) | |||
| Mapped | - | - | Status.JVM.Memory.Mapped | MemoryUsed | TotalCapacity |
| JVM Metaspace |
taskmanager.memory.jvm- |
metaspace. |
size | memoryConfiguration. |
| jvmMetaspace | Status.JVM.Memory. |
| Metaspace | Used | Max |
| JVM Overhead |
...
| memoryConfiguration.jvmOverhead | - | - |
1 These are the configuration parameters used in the Flink configuration.
2 These are the Json paths to address the properties in the HTTP REST API response. Additionally, memoryConfiguration.totalFlinkMemory and totalProcessMemory are exposed through the REST API.
3 The metrics which are exposed through the TaskManager's metrics
...
Frontend Design (out-dated)
...