You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

 

Pre-0.8 the integration tests span over core/src/test, some of them resides in the unit.kafka, some others in other.kafka. In 0.9 we want to have a cleaner coverage of integration tests along with the unit tests.

Unit Test

 

For each low-level component of kafka server and client that is self-functional, such as bufferpool/sende of producer, memoryrecords of common, replica/partition-manager of server, we will have a corresponding unit test class in the test/unit/<package-name>/<component-name>.

kafka.common:

  • config: config properties can be read/write correctly, default value can be retrieved, config errors can be detected.
  • protocol: schema fields can be put/get by pos/name correctly, schema/arrays can be serialized/de-ser correctly.
  • record:
    • record fields can be read/write correctly, record functions (size, isValid, checksum) works correctly.
    • memory-records functions (append with/wo compression, etc) works correctly.
  • requests: request serialization/de-ser correctly, and non-null fields are defined.
  • metadata:
    • cluster/topic-partition/partition functions works correctly.
    • serializer works properly.
  • network:
    • networksend/receive functions work correctly.
    • byterbuffersend/receive functions work correctly.
    • selector functions work correctly (may need some MockChannels).

kafka.clients.producer

  • partitioner: logical correctness.
  • recordsend: functions work correctly (may need MockCallback).
  • bufferpool: logical correct under multi-thread, failure cases.
  • metadata: logical correct under multi-thread, failure cases.
  • recordaccumulator: functional correct for read/drain, etc; logical correct for append under multi-thread.
  • recordbatch: succeed/fail cases of functions (tryAppend, etc)
  • sender: processReadyPartitions function, handleXX functions, and close function correctness.

Integration Test

For integration tests we will categories the following modules, and for each module we will do integration tests for different scenarios (normal success cases, failure cases with different configs, etc)

admin

  • Topic tools
  • Partition reassignment
  • Leader election
  • Start/stop server and zookeeper

api

Producer:

  • Sync and async send
  • Send null key or null value
  • Send to one and multiple partitions
  • Failed send exception thrown and callback

Consumer:

  • Offset reset
  • Commit and fetch offset
  • Pool with/wo timeout
  • Callback functions
  • Rebalance

config

  • Invalid values

end2end

  • New-topic auto creation
  • Compression
  • Replication with/wo broker failure/shutdown
    • Replica consistency
    • Producer recordsend behavior
    • Consumer Coordinator migration
    • ISR correctness
    • Guarantee with ack property
  • Producer message size
  • Consumer fetch size
  • Server (re)startup
    • Log recovery and cleaning
    • Consumer Coordinator migration

logging

  • String encoder
  • Appender property

util

 

 

 

  • No labels