This page is meant as a template for writing a FLIP. To create a FLIP choose Tools->Copy on this page and modify with your content and replace the heading with the next FLIP number and a description of your issue. Replace anything in italics with your own description.
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
FLIP-386 is building on top of FLIP-384. The intention here is to add a capability for state backends to attach custom attributes during recovery to recovery spans. For example RocksDBIncrementalRestoreOperation could report both remote download time and time to actually clip/ingest the RocksDB instances after rescaling.
Public Interfaces / Proposed Changes
This flip proposes to add OpenTelemetryMetricReporterFactory and OpenTelemetryTraceReporterFactory factories, that would be both available in the newly added plugin flink-metrics/flink-metrics-otel (similar structure as pre-existing MetricReporters like flink-metrics-jmx).
The only custom configuration will be:
exporter.endpoint - url of the OpenTelemetry endpoint
exporter.timeout - timeout when reporting to the endpoint (Default value
Both metrics and traces reporters will also support scope.variables.additional
Example configuration:
metrics.reporters: otel metrics.reporter.otel.factory.class: org.apache.flink.common.metrics.OpenTelemetryMetricReporterFactory metrics.reporter.otel.exporter.endpoint: http://127.0.0.1:1337 metrics.reporter.otel.scope.variables.additional: region:eu-west-1,environment:local-pnowojski-test,flink_runtime:1.17.1 traces.reporters: otel traces.reporter.otel.factory.class: org.apache.flink.common.metrics.OpenTelemetryTraceReporterFactory traces.reporter.otel.exporter.endpoint: http://127.0.0.1:1337 traces.reporter.otel.scope.variables.additional: region:eu-west-1,environment:local-pnowojski-test,flink_runtime:1.17.1
Compatibility, Deprecation, and Migration Plan
There will be no impact on existing users, there is no need for any migration.
Test Plan
On top of automated tests, this feature is already used and tested inside Confluent. Before committing the final accepted version would be tested inside Confluent again.
Rejected Alternatives
None