Status
Page properties | |||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Page properties | |||||||||||||||||||||||||
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
The Path class is currently mutable to support the IOReadableWritable serialization. However, but many parts of the code assume that the Path is immutable. The mutable Path class is prone to subtle errors. By making the URI in the Path class immutable, we can properly store configured paths without the risk of them being mutated as side effectsensure that paths are stored correctly without the possibility of mutation and eliminate the occurrence of subtle errors.
Public Interfaces
Modify the Path class to no longer implement the IOReadableWritable interface.
Add two static methods to the Path class to support serializing to DataInputView and deserializing from DataOutputView:public static Path deserializeFromDataInputView(DataInputView in) throws IOException
public static void serializeToDataOutputView(Path path, DataOutputView out) throws IOException
...
First, add the two static methods to the Path class to support serialization and deserialization. Currently, three classes need to serialize/deserialize the Path using the IOReadableWritable interface: FileSourceSplitSerializer/TestManagedSinkCommittableSerializer/TestManagedFileSourceSplitSerializer. Modify these classes to serialize/deserialize the Path using the new two static methods instead of IOReadableWritable.
...
- What impact (if any) will there be on existing users?
For the users who rely on IOReadableWritable for serialization/deserialization to serialize/deserialize the Path will need to migrate to using the serializeToDataOutputView/deserializeFromDataInputView methods in the Path class. This migration shouldn't be difficult.
- As the implementation method of IOReadableWritable in the Path class is part of the public API, we will deprecate the method 2 releases before removing it.