Versions Compared

Old Version 10

changes.mady.by.user Matthias Pohl

Saved on

compared with

New Version 11

changes.mady.by.user Matthias Pohl

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Refactored getAvailableMemorySize() to be getUsedMemorySize()

...

  1. add shuffle memory's size metric
    Keep in mind that we have to convert available memory into used memory somewhere later on!

    Code Block
    languagejava
    titleAdd network memory members
    public long getTotalMemorySize() {
	return 1L * getTotalNumberOfMemorySegments() * memorySegmentSize;
}

public long getAvailableMemorySizegetUsedMemorySize() {
	return getTotalMemorySize() - (getNumberOfAvailableMemorySegments() * memorySegmentSize);
}


  2. update NettyShuffleMetricFactory#registerShuffleMetrics

    Code Block
    languagejava
    private static final String METRIC_TOTAL_MEMORY_IN_BYTES = "TotalMemory";
private static final String METRIC_AVAILABLEUSED_MEMORY_IN_BYTES = "AvailableMemoryUsedMemory";

// ...

private static void registerShuffleMetrics(
		String groupName,
		MetricGroup metricGroup,
		NetworkBufferPool networkBufferPool) {
	MetricGroup networkGroup = metricGroup.addGroup(groupName);
	networkGroup.<Integer, Gauge<Integer>>gauge(METRIC_TOTAL_MEMORY_SEGMENT,
		networkBufferPool::getTotalNumberOfMemorySegments);	
	networkGroup.<Integer, Gauge<Integer>>gauge(METRIC_AVAILABLE_MEMORY_SEGMENT,
		networkBufferPool::getNumberOfAvailableMemorySegments);

	// === new ===
	networkGroup.<Long, Gauge<Long>>gauge(METRIC_TOTAL_MEMORY_IN_BYTES,
		networkBufferPool::getTotalMemorySize);
	networkGroup.<Long, Gauge<Long>>gauge(METRIC_AVAILABLEUSED_MEMORY_IN_BYTES,
		networkBufferPool::getAvailableMemorySizegetUsedMemorySize);
}


Step 2: Introduce new metrics for Task's managed memory usage

...