You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

Status

Current state: [In Progress]

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

JIRA:   Unable to render Jira issues macro, execution error.

Released: <Flink Version>

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Aligning the overview page of JobManager with the Flink's new memory model (FLIP-116).

Proposed Changes

This includes making JobManager's memory-related metrics available in the UI. Additionally, the configuration parameters should be exposed similar to the TaskManager's overview (see FLIP-102: Add More Metrics to TaskManager).

JVM Metrics

These JVM metrics are exposed and can be used through the metrics REST API.

JVMMetricUsed keyTotal key
HeapStatus.JVM.Memory.HeapUsedMax
DirectStatus.JVM.Memory.DirectUsedMax
MappedStatus.JVM.Memory.MappedMemoryUsedTotalCapacity
NonHeapStatus.JVM.Memory.NonHeapMemoryUsedTotalCapacity
Metaspace

Status.JVM.Memory.Metaspace Unable to render Jira issues macro, execution error.

UsedMax

Memory Configuration

Flink's memory model (as described in org.apache.flink.runtime.jobmanager.JobManagerProcessSpec) can be mapped to the following Flink configuration parameters. There are a few that have a correlating Flink metric.

Flink Memory ModelFlink configuration1REST API2MetricUsed keyTotal key
Heapjobmanager.memory.heap.size???Status.JVM.Memory.HeapUsedMax
Off-Heapjobmanager.memory.off-heap.size???---
JVM Metaspacejobmanager.memory.jvm-metaspace.size???

Status.JVM.Memory.Metaspace  Unable to render Jira issues macro, execution error.

UsedMax
JVM Overheadjobmanager.memory.jvm-overhead.min???---
jobmanager.memory.jvm-overhead.max

1 These are the configuration parameters used in the Flink configuration.
2 These are the Json paths to address the properties in the HTTP REST API response.  Unable to render Jira issues macro, execution error.

Frontend Design

Display metrics inside JVM, outside JVM and GC

REST API proposal:

Metrics

The API can be used to retrieve the metrics for the JobManager: http://localhost:8081/jobmanager/metrics?get=Status.JVM.Memory.Heap.Max,Status.JVM.Memory.Heap.Used,Status.JVM.Memory.NonHeap.Max,Status.JVM.Memory.NonHeap.Used,Status.JVM.Memory.Metaspace.Max,Status.JVM.Memory.Metaspace.Used

Memory Configuration


Test Plan

Covered by unit tests.

  • No labels