THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- A component to figure the optimal number of partitions to fetch data from source with an upper limit on the number of tasks and batch fetch size (rows per filetable).
- Query builder based on incremental logic configured by the user. Strategy types can be
- Incrementing column
- Timestamp column
- Timestamp and incrementing columns
- Customer query-based
- Full refresh
- Component to execute part operations independently with retry mechanism (spark map).
- Component to handle schema evolution and database-specific type conversions.
...