Build Engine Configuration

kylin.engine.spark.build-class-name

org.apache.kylin.engine.spark.job.CubeBuildJob

kylin.engine.spark.cluster-info-fetcher-class-name

org.apache.kylin.cluster.YarnInfoFetcher

kylin.engine.spark-conf.XXX

1. Before Kylin submit a cubing job, some major property(cores and memory) will be automatically adjusted adaptively. (if kylin.spark-conf.auto.prior was set to true).
2. After auto adjust, spark conf will be overwrite by this property. If you want to set spark.driver.extraJavaOptions=-Dhdp.version=current, you can add follow line in kylin.properties:

kylin.engine.spark-conf.spark.driver.extraJavaOptions=-Dhdp.version=current

kylin.storage.provider

org.apache.kylin.common.storage.DefaultStorageProvider

The content summary objects returned by different cloud vendors are not the same, so need to provide targeted implementation.

You can refer to this to learn more : org.apache.kylin.common.storage.IStorageProvider

kylin.engine.spark.merge-class-name

org.apache.kylin.engine.spark.job.CubeMergeJob

kylin.engine.spark.task-impact-instance-enabled

kylin.engine.spark.task-core-factor

kylin.engine.driver-memory-base

kylin.engine.driver-memory-strategy

{"2", "20", "100"}

kylin.engine.driver-memory-maximum

kylin.engine.persist-flattable-threshold

kylin.snapshot.parallel-build-timeout-seconds

kylin.snapshot.parallel-build-enabled

kylin.spark-conf.auto.prior

kylin.engine.submit-hadoop-conf-dir

Set HADOOP_CONF_DIR for spark-submit.

kylin.storage.columnar.shard-size-mb

The max size of pre-calcualted cuboid parquet file.

kylin.storage.columnar.shard-rowcount

2500000

kylin.storage.columnar.shard-countdistinct-rowcount

kylin.query.spark-engine.join-memory-fraction