You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Document the state by adding a label to the FLIP page with one of "discussion", "accepted", "released", "rejected".

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

The Path class is currently mutable to support the IOReadableWritable serialization, but many parts of the code assume that the Path is immutable. The mutable Path class is prone to subtle errors. By making the URI in the Path class immutable, we can properly store configured paths without the risk of them being mutated as side effects. 

Public Interfaces

Modify the Path class to no longer implement the IOReadableWritable interface.

Add two static methods to the Path class to support serializing to DataInputView and deserializing from DataOutputView:

public static Path deserializeFromDataInputView(DataInputView in) throws IOException

public static void serializeToDataOutputView(Path path, DataOutputView out) throws IOException

Proposed Changes

There are two steps to modify the Path class to no longer implement the IOReadableWritable interface.

First, add the two static methods to the Path class to support serialization and deserialization. Currently, three classes need to serialize/deserialize the Path using the IOReadableWritable interface:

FileSourceSplitSerializer/TestManagedSinkCommittableSerializer/TestManagedFileSourceSplitSerializer. Modify these classes to serialize/deserialize the Path using the two static methods instead of IOReadableWritable.

Second, mark the implementation methods of IOReadableWritable in the Path class as deprecated and remove them in the next major release.

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?
  • For the users who rely on IOReadableWritable for serialization/deserialization will need to migrate to using the serializeToDataOutputView/deserializeFromDataInputView methods in the Path class. This migration shouldn't be difficult.

  • As the implementation method of IOReadableWritable in the Path class is part of the public API, we will deprecate the method 2 releases before removing it.
  • No labels