Page History

...

the inputs only contains Tsignal@ds=1, but is should also contain T@ds=1/hr=1, T@ds=1/hr=2,....T@ds=1/hr=24

This dependency should be captured by the metastore. For simplicity, let us assume we create a new notion of dependent tables (instead of overloading external tables).

create dependency table Tdependent (key string, value string) partitioned by (ds string);

This is like a external table but also captures the dependency (we can also enhance external tables for the same)The idea is to let the user specify and store the dependency explicitly.

alter table Tsignal Tdependent add partition (ds='1') location 'Location of /T'/ds=1 depends ' dependent partitions table T partition partitions (ds='1');
specify the partial partition spec for the dependent partitions.
Note that each table can point to different locations - hive needs to ensure that all the dependent partitions are under the location 'T/ds=1'

The metastore can store the dependencies completely or partially.

• Materialize the dependencies both-ways
  Tdependent@ds=1 depends on T@ds=1/hr=1 to T@ds=1/hr=24