THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Code Block |
---|
Consumer (Properties) { // Subscribe to a list of topics, return immediately. // Throws an exception if the new subscription is not valid with existing subscriptions. void subscribe(String...) throws SubscriptionNotValidException; // Subscribe to a specified topic partition, return immediately. // Throws an exception if the new subscription is not valid with existing subscriptions. void subscribe(String, Int) throws SubscriptionNotValidException; // Subscribe to a specified topic partition at the given offset, return immediately. // Throws an exception if the new subscription is not valid with existing subscriptions. void subscribe(String, Int, Long) throws SubscriptionNotValidException; // Try to fetch messages from the topic/partitions it subscribed to. // Return whatever messages are available to be fetched or empty after specified timeout. List<Record> poll(Long) throws SubscriptionIsEmptyException, ConsumeInitOffsetNotKnownException; // Commit latest offsets for all partitions currently consuming from. // If the sync flag is set to true, commit call will be sync and blocking. // Otherwise the commit call will be async and best-effort. void commit(Boolean) throws SubscriptionIsEmptyException; // Commit the specified offset for the specified topic/partition. // If the sync flag is set to true, commit call will be sync and blocking. // Otherwise the commit call will be async and best-effort. // Throws an exception if the partitions specified are not currently consuming from. void commit(List[(String, Int, Long)], Boolean) throws InvalidPartitionsToCommitOffsetException; // Specify the fetch starting offset for the specified topic/partition. void pos(List[(String, Int, Long)]) // Get the currently consuming partitions. // Block wait if the current partitions are not known yet. List[(String, Int)] getPartitions() throws SubscriptionIsEmptyException; // Get the last committed offsets of the partitions currently consuming. // Block wait if the current partitions are not known yet. Map[(String, Int), Long] getOffsets() throws SubscriptionIsEmptyException; // Get the last committed offsets of the specified topic/partition. Long getOffset(String, Int) throws SubscriptionIsEmptyException, InvalidPartitionsToGetOffsetException; // --------- Call-back Below, not part of API ------------ // // Call-back function upon partition de-assignment. // Default implementation will commit offset depending on auto.commit config. void onPartitionDeassigned(List[(String, Int)]); // Call-back function upon partition re-assignment. // Default implementation is No-Op. void onPartitionAssigned(List[(String, Int)]) // --------- Optional Call-back Below ------------ // // Call-back function upon partition reassignment given the group member list and the subscribed topic partition info. // Return the partitions that would like to consume, and need to make sure each partition is covered by just one member in the group. List[(String, Int)] partitionsToConsume(List[String], List[TopicPartition]); } |
The Records returned by the poll() function will include metadata such as offset, partition, key and partition-key. This would also help removing the decompression/recompression in mirror maker (details can be found in KAFKA-1011).
...