Page History

...

For CatalogBaseTable, we use CatalogPropertiesUtil to serialize/deserialize it , it's the tools that Flink already provides.

The complexity of serialization Catalog is high, and we need to introduce a new serialization mechanism to achieve this, which is described in a separate section.

Catalog Serialization Solutions :

Option 1: Serialize the options in the Create Catalog DDL

We For Catalog, we need to serialize catalog name and the options which are used in create catalog DDL, then JM side can use these options to re-initialize the catalog by flink ServiceLoader mechnism(UsingFactoryUtil#createCatalog to get catalog).  To InMemoryCatalog, here are some special case. Due to the tables in InMemoryCatalog already exist in the external system, metadata information in InMemoryCatalog is only used by the job itself and is only stored in memory. Therefore, all metadata The database related information in InMemoryCatalog does not need needs to be serialized and then passed to JM, so this solution can cover the usage scenario of InMemoryCatalog.otherwise the database may not exist when JM creates the table. 

Here we give an example about catalog serializable process that catalog is created by DDL way.

...