Versions Compared

Old Version 2

changes.mady.by.user Dian Fu

Saved on

compared with

New Version 3

changes.mady.by.user Dian Fu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The signature of from_pandas is as following:

class TableEnvironment(object):

    def from_pandas(self, pdf: pd.DataFrame, schema: Union[RowType, list, tuple] = None) -> Table:

        pass


The argument schema is used to specify the schema of the result table:

...

The signature of to_pandas is as following:

class Table(object):

    def to_pandas(self) -> pd.DataFrame:

        pass


Proposed Changes

from_pandas

...

  1. The parallelism of the ArrowTableSink could only be 1.

Example

# Suppose there is already a Pandas dataframe
pdf = xxx
...

row_type = DataTypes.ROW(

    [DataTypes.FIELD("f1", DataTypes.INT()),

     DataTypes.FIELD("f2", DataTypes.FLOAT()),

     DataTypes.FIELD("f3", DataTypes.BOOLEAN()),

     DataTypes.FIELD("f4", DataTypes.STRING())])

# convert the Pandas dataframe to Flink table
table = t_env.from_pandas(pdf, row_type)

# perform some operations on the Flink table
filtered_table = table.filter("f1 > 0")

# convert the result Flink table to Pandas dataframe
filtered_pdf = filtered_table.to_pandas()


Compatibility, Deprecation, and Migration Plan

...