THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Table of Contents |
|---|
1. Controlled Shutdown
What is controlled shutdown?
...
How to use the tool?
| Code Block |
|---|
bin/kafka-run-class.sh kafka.admin.ShutdownBroker --zookeeper localhost:12913/kafka --broker #brokerId# --num.retries 3 --retry.interval.ms 60
|
...
How to use the tool?
| Code Block |
|---|
bin/kafka-preferred-replica-election.sh --zookeeper localhost:12913/kafka --path-to-json-file topicPartitionList.json
|
...
Example json file (This is optional. This can be specified to move the leader to the preferred replica for specific topic partitions)
| Code Block |
|---|
{
"partitions":
[
{"topic": "topic1", "partition": "0"},
{"topic": "topic1", "partition": "1"},
{"topic": "topic1", "partition": "2"},
{"topic": "topic2", "partition": "0"},
{"topic": "topic2", "partition": "1"},
]
}
|
...
How to use the tool?
| Code Block |
|---|
List info for topic1
bin/kafka-list-topic.sh --zookeeper localhost:2121 --topic topic1
List info for all topics
bin/kafka-list-topic.sh --zookeeper localhost:2121
List info for topics which have under replicated count
bin/kafka-list-topic.sh --zookeeper localhost:2121 --under-replicated-partitions
List info for topics whose leader for a partition is not available
bin/kafka-list-topic.sh --zookeeper localhost:2121 --unavailable-partitions
|
...
How to use the tool?
| Code Block |
|---|
bin/kafka-create-topic.sh
Option Description
------ -----------
--partition <Integer: # of partitions> number of partitions in the topic
(default: 1)
--replica <Integer: replication factor> replication factor for each partitions
in the topic (default: 1)
--replica-assignment-list for manually assigning replicas to brokers
(default: )
<broker_id_for_part1_replica1 :
broker_id_for_part1_replica2,
broker_id_for_part2_replica1 :
broker_id_for_part2_replica2, ...>
--topic <topic> REQUIRED: The topic to be created.
--zookeeper <urls> REQUIRED: The connection string for
the zookeeper connection in the form
host:port. Multiple URLS can be
given to allow fail-over.
|
...
How to use the tool?
| Code Block |
|---|
bin/kafka-add-partitions.sh
Option Description
------ -----------
--partition <Integer: # of partitions> REQUIRED: Number of partitions to add
to the topic
--replica-assignment-list For manually assigning replicas to
<broker_id_for_part1_replica1 : brokers for the new partitions
broker_id_for_part1_replica2, (default: )
broker_id_for_part2_replica1 :
broker_id_for_part2_replica2, ...>
--topic <topic> REQUIRED: The topic for which
partitions need to be added.
--zookeeper <urls> REQUIRED: The connection string for
the zookeeper connection in the form
host:port. Multiple URLS can be
given to allow fail-over.
|
...
This tool is only available in the 0.8 branch for now.
How to use the tool?
WARNING: The tool was released in beta in 0.8 and has some bugs that can render the topic unusable. The tool is known to be stable in 0.8.1.
| Code Block |
|---|
bin/kafka-reassign-partitions.sh
Option Description
------ -----------
--broker-list <brokerlist> The list of brokers to which the
partitions need to be reassigned in
the form "0,1,2". This is required
for automatic topic reassignment.
--execute [execute] This option does the actual
reassignment. By default, the tool
does a dry run
--manual-assignment-json-file <manual The JSON file with the list of manual
assignment json file path> reassignmentsThis option or topics-
to-move-json-file needs to be
specified. The format to use is -
{"partitions":
[{"topic": "foo",
"partition": 1,
"replicas": [1,2,3] }],
"version":1
}
--topics-to-move-json-file <topics to The JSON file with the list of topics
reassign json file path> to reassign.This option or manual-
assignment-json-file needs to be
specified. The format to use is -
{"topics":
[{"topic": "foo"},{"topic": "foo1"}],
"version":1
}
--zookeeper <urls> REQUIRED: The connection string for
the zookeeper connection in the form
host:port. Multiple URLS can be
given to allow fail-over.
|
...
The following example moves 2 topics (foo1, foo2) to newly added brokers in a cluster (5,6,7)
| Code Block |
|---|
nnarkhed$ ./bin/kafka-reassign-partitions.sh --topics-to-move-json-file topics-to-move.json --broker-list "5,6,7" --execute
nnarkhed$ cat topics-to-move.json
{"topics":
[{"topic": "foo1"},{"topic": "foo2"}],
"version":1
}
|
...
The following example moves 1 partition (foo-1) from replicas 1,2,3 to 1,2,4
| Code Block |
|---|
nnarkhed$ ./bin/kafka-reassign-partitions.sh --manual-assignment-json-file partitions-to-move.json --execute
nnarkhed$ cat partitions-to-move.json
{"partitions":
[{"topic": "foo",
"partition": 1,
"replicas": [1,2,4] }],
}],
"version":1
}
|
...
How to use the tool ?
| Code Block |
|---|
nnarkhed-mn:kafka-git-idea nnarkhed$ ./bin/kafka-run-class.sh kafka.tools.StateChangeLogMerger
Provide arguments to exactly one of the two options "[logs]" or "[logs-regex]"
Option Description
------ -----------
--end-time <end timestamp in the The latest timestamp of state change
format java.text. log entries to be merged (default:
SimpleDateFormat@f17a63e7> 9999-12-31 23:59:59,999)
--logs <file1,file2,...> Comma separated list of state change
logs or a regex for the log file
names
--logs-regex <for example: /tmp/state- Regex to match the state change log
change.log*> files to be merged
--partitions <0,1,2,...> Comma separated list of partition ids
whose state change logs should be
merged
--start-time <start timestamp in the The earliest timestamp of state change
format java.text. log entries to be merged (default:
SimpleDateFormat@f17a63e7> 0000-00-00 00:00:00,000)
--topic <topic> The topic whose state change logs
should be merged
|