You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Status

Current state: Under Discussion

Discussion thread:

JIRA:  FLINK-25756 - Dedicated Opensearch connectors

Released: <Flink Version>

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Since the time Opensearch got forked from Elasticsearch a few things got changed. The projects evolve in different directions, the Elasticsearch clients up to 7.13.x were able to connect to Opensearch clusters, but since 7.14 - not anymore (Elastic continues to harden their clients to connect to Elasticsearch clusters only, https://github.com/elastic/elasticsearch/pull/73910). Arguably, the best way to proceed in this situation is to provide dedicated Opensearch connectors, on par with Elasticsearch ones.

Proposal

We propose to introduce dedicated Opensearch connectors (for 1.x and 2.x release lines) in Flink, based on existing Elasticsearch 7 connectors (https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/datastream/elasticsearch/).

Scope

The  Opensearch connectors will support:

  • Integrate with Flink Sink API (FLIP-143).
  • Build upon Flink New DynamicTableSink and DynamicTableSinkFactory interfaces (FLIP-95)


  • No labels