Page History

...

• 自动调参功能是默认打开的
• 仅仅会在集群模式生效
• 会影响的spark的配置项如下(当且仅当用户没有在kylin.properties中配置如下属性的时候才会启用本功能)

  
  

  Code Block
  # Spark executor kylin.engine.spark-conf.spark.executor.instances kylin.engine.spark-conf.spark.executor.cores kylin.engine.spark-conf.spark.executor.memory kylin.engine.spark-conf.spark.executor.memoryOverhead # Spark driver kylin.engine.spark-conf.spark.driver.memory # Shuffle kylin.engine.spark-conf.spark.sql.shuffle.partitions

  
  

实现

Spark driver参数

driver端只会修改spark.driver.memory, 执行逻辑如下； Spark任务会使用如下公式min(kylin.engine.driver-memory-maximum, kylin.engine.driver-memory-base * 档位)

...