...

Table-3: Avg response time of TPC-H Query (sf=100)

Conclusions

Query performance.

In big query scenarios(query which scans and does onsite complex calculations on a large mount of partitions/files) which use TPCH-100, response time of Kylin 4 on S3 with Soft Affinity and Local Cache has significant less than kylin 4 on S3 only.

Thanks to Soft Affinity and Local Cache, Kylin 4 query performance improvements can be achieved in basically most queries.

It is observed that the results (Q4, Q13) of turning on the Soft Affinity and Local Cache are lower than when using S3 alone as storage. This may be due to some reason that the data was not read through the cache. The underlying reason was not carried out in this test. Further analysis, we will gradually improve in the subsequent optimization process.

On the conclusion, Soft Affinity and Local Cache can achieve significant performance improvements for both simple and complex queries.