FAQ Kylin 4.X

If you did not find the answer of your question, feel free to ask in user's mailing list.

---

Question List

• How do RowKey affect storage?

To be updated

• What is Sparder(SparderContext)? And how should I take care of it ?

                SparderContext is the implemenatation of new distributed query engine which is backend by a spark application. If SparderContext is dead, all your query will failed. So please check its liveness after Kylin instance(Query Server) was started in application list of Resource Manager Web UI. (We plan to add canary tools for monitor Sparder in 4.0.0-beta)

• Is Hadoop3 supported ?

                Kylin 4.0.0-alpha did not support Hadoop3. 

• If you faced Exception with message : "Cannot find hive-site.xml in kylin_hadoop_conf_dir", please:

1. Copy all files under /etc/hadoop/conf to one directory ("/path/to/hadoop_conf").

2. Copy hive-site.xml to "/path/to/hadoop_conf".

3. Edit kylin.properties, modify kylin.env.hadoop-conf-dir=/path/to/hadoop_conf, restart Kylin.

• How to achieve Read/Write Separation Deployment？

                Please refer to Read Write Separation Deployment for Kylin 4.0.

• How to refresh the lookup table snapshot?

                It will be automatically refreshed the next time build.

• How to use the new garbage cleaning tool, which garbage will be cleaned up?

                To be updated

• Can Cube Planner be used?

                Not currently supported.

• Where is the dimension dictionary stored?

                Dimension dictionary is removed.

• What are the optimization methods for build engine?

                To be updated.

• What are the optimization methods for query engine?

                Please refer to Improve query performance by setting shard by column.

• Is Kylin 3.x and Kylin 4.x metadata compatible?

                Fully compatible.

• Is Kylin 3.x and Kylin 4.x cuboid data compatible? If not, will there be a migration plan? 

                The pre-calculated cuboid data is completely incompatible, and there is no migration plan for the time being, due to relatively large effort in development.

• Is the Spark used by Kylin the community version?

                Spark 2.4.6 is currently supported.

• What features will no longer be supported in Kylin 4?

                Please refer to Kylin 4.X Feature List.

• What is the performance of query engine and build engine in Kylin 4?

                To be updated

• Will query results in Kylin 4 be consistent with the previous version? 

                To be updated

• "Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Please reduce your request rate. (Service: Amazon S3; Status Code: 503; Error Code: SlowDown;"
  • https://aws.amazon.com/premiumsupport/knowledge-center/emr-s3-503-slow-down/
  • We can reverse the Kylin's working dir.
• How to develop UDF and UDAF？

                To be updated

• Does Kylin 4 support AWS Glue?

                To be updated

• Does query on Spark support Spark Schduler Pool setting(resource isolation)?  

                Use different spark pool for different query

• What is the implementation of the new global dictionary?

                Please refer to Global Dictionary on Spark.

• How to store Metadata after not using HBase?

                Kylin supports JDBC to store Metadata, please refer to Use MySQL as Metastore and How to use HBase metastore in Kylin 4.0.