...
Page properties | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Document the state by adding a label to the FLIP page with one of "discussion", "accepted", "released", "rejected".
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
Original | Class | Key | Deprecated Key | Default Value | Type | Description |
TASK_MANAGER_LOG_PATH_KEY | TaskManagerOptions | taskmanager.log.path | System.getProperty("log.file") | String | The path to the log file of the taskmanager. | |
FS_STREAM_OPENING_TIMEOUT_KEY | TaskManagerOptions | taskmanager.runtime.fs-timeout | taskmanager.runtime.fs_timeout | 0 ms | Duration | The timeout for filesystem stream opening. A value of 0 indicates infinite waiting. |
YARN_CONTAINER_START_COMMAND_TEMPLATE | YarnConfigOptions | yarn.container-start-command-template | %java% %jvmmem% %jvmopts% %logging% %class% %args% %redirects% | String | Template for the YARN container start invocation. | |
LOCAL_NUMBER_TASK_MANAGER | TaskManagerOptions | minicluster.number-of-taskmanagertaskmanagers | local.number-taskmanager | 1 | Integer | The number of task manager of MiniCluster. |
...
LOCAL_NUMBER_JOB_MANAGER
DEFAULT_LOCAL_NUMBER_JOB_MANAGER
HA_JOB_MANAGER_PORT
DEFAULT_EXECUTION_RETRIES
DEFAULT_FILESYSTEM_SCHEME
DEFAULT_FILESYSTEM_OVERWRITE
DEFAULT_STATE_BACKEND
ENV_FLINK_BIN_DIR
2.1.3 Update FileInputFormat.java, FileOutputFormat.java, BinaryInputFormat.java, and BinaryOutputFormat.java to deprecate
...
string configuration keys:
...
- FileInputFormat.java
...
...
Class
...
Key
...
Deprecated Key
...
Default Value
...
Type
...
Description
FILE_PARAMETER_KEY
...
InputOutputFormatOptions
...
input-format.file.path
...
input.file.path
...
...
String
...
The input file path.
- ENUMERATE_NESTED_FILES_FLAG
...
...
input-format.file.recursive.enabled
...
recursive.file.enumeration
...
false
...
Boolean
...
Whether input directories are recursively traversed.
...
- FileOutputFormat.java
...
Original
...
Class
...
Key
...
Deprecated Key
...
Default Value
...
Type
- FILE_PARAMETER_KEY
...
InputOutputFormatOptions
...
...
output.file.path
...
none
...
String
...
The output file path.
- BinaryInputFormat.java
...
Original
...
Class
...
Key
...
Deprecated Key
...
Default Value
...
Type
...
Description
- BLOCK_SIZE_PARAMETER_KEY
...
InputOutputFormatOptions
...
input-format.binary.block_size
...
input.block_size
...
Long.MIN_VALUE
...
...
The fixed length of a record.
...
- BinaryOutputFormat.java
...
Original
...
Class
...
Key
...
Deprecated Key
...
Default Value
...
Type
- BLOCK_SIZE_PARAMETER_KEY
...
InputOutputFormatOptions
...
output-format.binary.block_size
...
output.block_size
...
Long.MIN_VALUE
...
Long
...
2.2 Public interfaces part2: Simplify the Configuration
...
- Deprecate getXXX(String key, XXX defaultValue) and setXXX(String key, XXX value), discussed in the preview thread, except the following:
- getString(String key, String defaultValue)
- setString(String key, String value)
- getBytes(String key, byte[] defaultValue)
- will be marked as @Internal
- setBytes(String key, byte[] bytes)
- Discussed at this preview thread: https://lists.apache.org/thread/zzsf7glfcdjcjm1hfo1xdwc6jp37nb3m
- will be marked as @Internal
- Update the comment in both getString(String key, String defaultValue) and setString(String key, String value) to encourage users to use ConfigOption.
...
Code Block | ||||
---|---|---|---|---|
| ||||
@Public public class Configuration extends ExecutionConfig.GlobalJobParameters implements IOReadableWritable, java.io.Serializable, Cloneable, ReadableConfig, WritableConfig { ... /** * Returns the value associated with the given key as a string. Getting value with string key is We encourage users and * developers to always use ConfigOption for getting the configurations if possible, for its * rich description, type, default-value and other supports. The string-key-based getter should * discouraged. Please use {@link #get(ConfigOption)} or {@link #getOptional(ConfigOption)} only be used when ConfigOption is not applicable, e.g., the key is programmatically generated * in runtime. * * @param key the key pointing to the associated value * @param defaultValue the default value which is returned in case there is no value associated * with the given key * @return the (default) value associated with the given key */ public String getString(String key, String defaultValue) { ... } /** * Adds the given key/value pair to the configuration object. Setting value with string key is We encourage users and developers * to always use ConfigOption for setting the configurations if possible, for its rich * description, type, default-value and other supports. The string-key-based setter should only * discouraged. Please use {@link #set(ConfigOption, Object)} be used when ConfigOption is not applicable, e.g., the key is programmatically generated in * runtime. * * @param key the key of the key/value pair to be added * @param value the value of the key/value pair to be added */ public void setString(String key, String value) { ... } ... } |
2.2.2 Introduce public <T> T get(ConfigOption<T> configOption, T overrideDefault)
Code Block |
---|
/**
* Returns the value associated with the given config option as a T. If no value is mapped
* under any key of the option, it returns the specified default instead of the option's default
* value.
*
* @param configOption The configuration option
* @param overrideDefault The value to return if no value was mapper for any key of the option
* @return the configured value associated with the given config option, or the overrideDefault
*/
@PublicEvolving
public <T> T get(ConfigOption<T> configOption, T overrideDefault) {
return getOptional(configOption).orElse(overrideDefault);
} |
`public <T> T get(ConfigOption<T> option, T overrideDefault)` can replace all old `getXxx(ConfigOption<Xxx> configOption, Xxx overrideDefault)` methods.
2.2.3 Deprecate some unnecessary setXxx and getXxx methods in Configuration
Changes:
- Mark all
public Xxx getXxx(ConfigOption<Xxx> configOption)
methods as @Deprecated, and remove them in 2.0. Such as:- public int getInteger(ConfigOption<Integer> configOption)
- public String getString(ConfigOption<String> configOption)
- public long getLong(ConfigOption<Long> configOption)
- public long getFloat(ConfigOption<Float> configOption)
- public long getDouble(ConfigOption<Double> configOption)
- public boolean getBoolean(ConfigOption<Boolean> configOption)
- Mark all
public void setXxx(ConfigOption<Xxx> key, Xxx value)
methods as @Deprecated, and remove them in 2.0. Such as:- public void setInteger(ConfigOption<Integer> key, int value)
- public void setString(ConfigOption<String> key, String value)
- public void setLong(ConfigOption<Long> key, long value)
- public void setDouble(ConfigOption<Double> key, double value)
- public void setFloat(ConfigOption<Float> key, float value)
- public void setBoolean(ConfigOption<Boolean> key, boolean value)
- Mark all
public Xxx getXxx(ConfigOption<Xxx> configOption, Xxx overrideDefault)
methods as @Deprecated, and remove them in 2.0. Such as:- public String getString(ConfigOption<String> configOption, String overrideDefault)
- public long getLong(ConfigOption<Long> configOption, long overrideDefault)
- public int getInteger(ConfigOption<Integer> configOption, int overrideDefault)
- public double getDouble(ConfigOption<Double> configOption, double overrideDefault)
- public float getFloat(ConfigOption<Float> configOption, float overrideDefault)
- public boolean getBoolean(ConfigOption<Boolean> configOption, boolean overrideDefault)
Reason:
- Configuration has a
public <T> T get(ConfigOption<T> option)
andpublic <T> Configuration set(ConfigOption<T> option, T value)
method - These getXxx and setXxx methods can be replaced with get and set directly without any extra efforts.
- get and set methods are easier to use than getXxx and setXxx
- Callers can call get directly, and users or flink developers don't need to care about should they call getInteger or getString.
- Flink code is easier to maintain.
T get(ConfigOption<T> option)
is designed later thanXxx getXxx(ConfigOption<Xxx> configOption)
, I guess ifT get(ConfigOption<T> option)
is designed first, allXxx getXxx(ConfigOption<Xxx> configOption)
methods aren't needed.
...