This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: Under Discussion

Discussion thread: Here

JIRA: KAFKA-1646

Motivation

Currently, when create on LogSegment, always create on empty file and keep APPEND data to it, for Linux file system (ext2/ext3/ext4 etc), it works fine.   But for windows and some old unix/linux file system, after a while, there will be more and more fragments on hard disk, and affect consume performance a lot. So if we pre allocate file with one bigger value ( for example, 512MB) when create file, it will help us reduce fragments on hard disk and improve consume performance.

Public Interfaces

This proposal add one config.

log.preallocate  - Should pre allocate file when create new segment?  Default value is false for backward compatible.

Proposed Changes

1. Configuration - add one configuration item "log.preallocate",  parse it in KafkaConfig.scala, and transfer to KafkaServer.scala, LogConfig.scala.
2. In Log.scala 
   1.  add one function initFileSize().  if the log preallocation is enabled, the value is config.segmentSize - 2 * config.maxMessageSize,  else the value is 0.
   2.  pass the initFileSize and config.preallocate to LogSegment when need create one LogSegment.
   3.  When need roll to a new file, trim log file of active segment
3. In LogSegment.scala - pass the initFileSize and preallocation to FileMessageSet.
4. In FileMessageSet.scala 
   1.  add one function trim().  Call it when close or roll a new LogSegment.
   2. Move openChannel from CoreUtil.scala and add two more parameter "initFileSize" and "preallocate".
   3. Add one constructor function with two more parameters  "initFileSize" and "preallocate".
5. In CoreUtils.scala
   1. Move openChallel to FileMessageSet.scala

Compatibility, Deprecation, and Migration Plan

This change is fully backward compatible.

Rejected Alternatives