Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Something that can be achieved by external tables currently. The dependent partitions are computed dynamically - T1@ds=1/hr=1 does not know the fact that it is dependent upon by Tdependent@ds=1.
T1@ds=1/hr=1 can be dropped anytime, and Tdependent@ds=1 automatically stops depending upon it from that point.

I am leaning towards this - the user need not specify both the location and the dependent partitions.
Can the external tables be enhanced to support this ? Will create a problem for the query layer, since the external tables are handled differently today.

    • The list of dependent partitions are materialized and stored in the metastore, and use that for querying.
      A query like 'select .. from Tdependent where ds = 1' gets transformed to 'select .. from (select * from T where ((ds = 1 and hr = 1) or (ds = 1 and hr = 2) .... or (ds=1 and hr=24))'
      Can put a lot of load on the query layer.