...
Something that can be achieved by external tables currently. The dependent partitions are computed dynamically - T1@ds=1/hr=1 does not know the fact that it is dependent upon by Tdependent@ds=1.
T1@ds=1/hr=1 can be dropped anytime, and Tdependent@ds=1 automatically stops depending upon it from that point.
I am leaning towards this - the user need not specify both the location and the dependent partitions.
Can the external tables be enhanced to support this ? Will create a problem for the query layer, since the external tables are handled differently today.
- The list of dependent partitions are materialized and stored in the metastore, and use that for querying.
A query like 'select .. from Tdependent where ds = 1' gets transformed to 'select .. from (select * from T where ((ds = 1 and hr = 1) or (ds = 1 and hr = 2) .... or (ds=1 and hr=24))'
Can put a lot of load on the query layer.
- The list of dependent partitions are materialized and stored in the metastore, and use that for querying.