Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Page properties

Document the state by adding a label to the FLIP page with one of "discussion", "accepted", "released", "rejected".

JIRA
Discussion thread

preview discussion : https://lists.apache.org/thread/zzsf7glfcdjcjm1hfo1xdwc6jp37nb3m

Official discussion: https://lists.apache.org/thread/zfw1b1g3679yn0ppjbsokfrsx9k7ybg0

Vote threadhttps://lists.apache.org/thread/joyr7bxpo0lcj1zfzdj5nv0lrhb303rx
JIRA

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

keyFLINK-34079

Release1.19, 1.20, 2.0


Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

Original

Class

Key

Deprecated Key

Default Value

Type

Description

TASK_MANAGER_LOG_PATH_KEY

TaskManagerOptions

taskmanager.log.path


System.getProperty("log.file")

String

The path to the log file of the taskmanager. 

FS_STREAM_OPENING_TIMEOUT_KEY

TaskManagerOptions

taskmanager.runtime.fs-timeout

taskmanager.runtime.fs_timeout

0 ms

Duration

The timeout for filesystem stream opening. A value of 0 indicates infinite waiting.

YARN_CONTAINER_START_COMMAND_TEMPLATE

YarnConfigOptions

yarn.container-start-command-template


%java% %jvmmem% %jvmopts% %logging% %class% %args% %redirects%

String

Template for the YARN container start invocation.

LOCAL_NUMBER_TASK_MANAGER

TaskManagerOptions

minicluster.number-of-taskmanagertaskmanagers

local.number-taskmanager

1

Integer

The number of task manager of MiniCluster.

...

  • Deprecate getXXX(String key, XXX defaultValue) and setXXX(String key, XXX value), discussed in the preview thread, except the following:
    • getString(String key, String defaultValue)
    ,
    • setString(String key, String value)
    ,
    • getBytes(String key, byte[] defaultValue)
    , and
    • will be marked as @Internal
    • setBytes(String key, byte[] bytes)
    .
    • will be marked as @Internal
  • Update the comment in both getString(String key, String defaultValue) and setString(String key, String value) to encourage users to use ConfigOption.

...

Code Block
languagejava
titleConfiguration.java
@Public
public class Configuration extends ExecutionConfig.GlobalJobParameters
        implements IOReadableWritable,
                java.io.Serializable,
                Cloneable,
                ReadableConfig,
                WritableConfig {
...      

	/**
     * Returns the value associated with the given key as a string. We encourage users and
     * developers to always use ConfigOption for getting the configurations if possible, for its
     * rich description, type, default-value and other supports. The string-key-based getter should
     * only be used when ConfigOption is not applicable, e.g., the key is programmatically generated
     * in runtime.
     *
     * @param key the key pointing to the associated value
     * @param defaultValue the default value which is returned in case there is no value associated
     *     with the given key
     * @return the (default) value associated with the given key
     */     
	public String getString(String key, String defaultValue) {
		...
    }


    /**
     * Adds the given key/value pair to the configuration object. We encourage users and developers
     * to always use ConfigOption for setting the configurations if possible, for its rich
     * description, type, default-value and other supports. The string-key-based setter should only
     * be used when ConfigOption is not applicable, e.g., the key is programmatically generated in
     * runtime.
     *
     * @param key the key of the key/value pair to be added
     * @param value the value of the key/value pair to be added
     */
    public void setString(String key, String value) {
        ...
    }
	
	...			
}

2.2.2 Introduce public <T> T get(ConfigOption<T> configOption, T overrideDefault)


Code Block
    /**
     * Returns the value associated with the given config keyoption as a byteT. array.If Weno encouragevalue usersis andmapped
     * developersunder toany alwayskey use ConfigOption for getting the configurations if possible, for its
     * rich description, type, default-value and other supports. The string-key-based getter should
     * only be used when ConfigOption is not applicable, e.g., the key is programmatically generatedof the option, it returns the specified default instead of the option's default
     * in runtimevalue.
     *
     * @param keyconfigOption The key pointing to the associated value.configuration option
     * @param defaultValueoverrideDefault The default value whichto is returned in case there isreturn if no value associated
was mapper for any key *     with of the given key.option
     * @return the (default)configured value associated with the given key.
config option, or the overrideDefault */
     public byte[] getBytes(String key, byte[] defaultValue) {
		...
    }

    /**   
	 */
     * Adds the given byte array to the configuration object. We encourage users and developers to@PublicEvolving
    public *<T> alwaysT use ConfigOption for setting the configurations if possible, for its rich description,get(ConfigOption<T> configOption, T overrideDefault) {
     * type, default-value and other supports. The string-key-based setter should only be used whenreturn getOptional(configOption).orElse(overrideDefault);
     * ConfigOption is not applicable, e.g., the key is programmatically generated in runtime.
     *
     * @param key The key under which the bytes are added.
     * @param bytes The bytes to be added.
     */
    public void setBytes(String key, byte[] bytes) {
        ...
    }

		...		
}}


`public <T> T get(ConfigOption<T> option, T overrideDefault)` can replace all old `getXxx(ConfigOption<Xxx> configOption, Xxx overrideDefault)` methods.

2.2.

...

3 Deprecate some unnecessary setXxx and getXxx methods in Configuration

Changes:

  • Mark all public Xxx getXxx(ConfigOption<Xxx> configOption)  methods as @Deprecated, and remove them in 2.0. Such as:
    • public int getInteger(ConfigOption<Integer> configOption)
    • public String getString(ConfigOption<String> configOption)
    • public long getLong(ConfigOption<Long> configOption)
    • public long getFloat(ConfigOption<Float> configOption)
    • public long getDouble(ConfigOption<Double> configOption)
    • public boolean getBoolean(ConfigOption<Boolean> configOption)
  • Mark all public void setXxx(ConfigOption<Xxx> key, Xxx value)  methods as @Deprecated, and remove them in 2.0. Such as:
    • public void setInteger(ConfigOption<Integer> key, int value)
    • public void setString(ConfigOption<String> key, String value)
    • public void setLong(ConfigOption<Long> key, long value)
    • public void setDouble(ConfigOption<Double> key, double value)
    • public void setFloat(ConfigOption<Float> key, float value)
    • public void setBoolean(ConfigOption<Boolean> key, boolean value)
  • Mark all public Xxx getXxx(ConfigOption<Xxx> configOption, Xxx overrideDefault)  methods as @Deprecated, and remove them in 2.0. Such as:
    • public String getString(ConfigOption<String> configOption, String overrideDefault)
    • public long getLong(ConfigOption<Long> configOption, long overrideDefault)
    • public int getInteger(ConfigOption<Integer> configOption, int overrideDefault)
    • public double getDouble(ConfigOption<Double> configOption, double overrideDefault)
    • public float getFloat(ConfigOption<Float> configOption, float overrideDefault)
    • public boolean getBoolean(ConfigOption<Boolean> configOption, boolean overrideDefault)

Reason:

  • Configuration has a public <T> T get(ConfigOption<T> option)   and public <T> Configuration set(ConfigOption<T> option, T value)  method
  • These getXxx and setXxx methods can be replaced with get and set directly without any extra efforts.
  • get and set methods are easier to use than getXxx and setXxx
    • Callers can call get directly, and users or flink developers don't need to care about should they call getInteger or getString.
  • Flink code is easier to maintain.
  • T get(ConfigOption<T> option)  is designed later than Xxx getXxx(ConfigOption<Xxx> configOption) , I guess if T get(ConfigOption<T> option)  is designed first, all Xxx getXxx(ConfigOption<Xxx> configOption)  methods aren't needed.

...