Versions Compared

Old Version 3

changes.mady.by.user Neha Narkhede

Saved on

compared with

New Version Current

changes.mady.by.user John Fung

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3
Table of Contents

It would be worthwhile to automate our performance testing to act as a generic integration test suite.

...

Ideally we would repeat this scenario and script in broker failures (kills), server pauses (simulated), etc.

0.8 Performance testing

Producer throughput

Message size : ~1K Production Data

Throughput in MB/s

 

 

Kafka Version

0.7

 

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.8

 

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.8

 

 

Replication Factor

n/a

 

1

1

1

1

1

1

1

1

1

 

2

2

2

2

2

2

2

2

2

 

 

Acks

n/a

 

-1

1

0

-1

1

0

-1

1

0

 

-1

1

0

-1

1

0

-1

1

0

 

 

Compression

Uncomp

 

Uncomp

Uncomp

Uncomp

Gzip

Gzip

Gzip

Snappy

Snappy

Snappy

 

Uncomp

Uncomp

Uncomp

Gzip

Gzip

Gzip

Snappy

Snappy

Snappy

Producer threads

Batch size

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1

1

 

29.57

 

1.66

1.69

19.48

0.94

0.97

3.51

1.53

1.46

11.40

 

0.56

1.61

23.52

0.35

0.91

3.70

0.56

1.45

10.83

2

1

 

45.35

 

3.31

3.11

23.82

1.80

1.72

3.87

2.30

2.96

11.65

 

1.27

2.79

20.59

0.66

1.39

3.86

1.07

2.19

11.39

5

1

 

58.53

 

5.24

5.49

20.26

2.59

2.86

3.35

4.02

4.48

13.36

 

1.79

4.76

21.29

1.16

2.49

3.06

1.93

3.98

10.49

10

1

 

50.23

 

8.20

8.59

19.65

3.17

2.77

2.87

7.35

7.39

10.51

 

2.90

7.97

19.44

1.85

2.68

3.05

3.27

7.24

12.01

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1

50

 

49.15

 

17.95

18.88

77.43

7.93

7.66

26.88

15.42

15.44

62.33

 

10.57

15.76

57.85

4.79

7.50

28.61

10.31

13.98

66.15

2

50

 

84.24

 

34.94

33.41

82.67

14.88

15.25

30.13

29.77

26.47

71.18

 

17.26

27.26

78.50

9.53

14.29

30.11

16.82

26.10

90.98

5

50

 

102.44

 

64.38

62.82

89.66

17.56

18.23

19.47

57.54

58.58

71.71

 

26.02

47.69

86.62

12.48

17.39

31.54

33.13

54.18

74.96

10

50

 

103.02

 

61.06

64.57

86.48

18.45

17.24

20.38

59.59

58.44

69.34

 

28.08

59.80

91.62

18.68

17.90

20.02

35.62

62.03

72.02

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1

100

 

49.76

 

23.69

23.56

75.71

8.93

9.04

28.69

21.73

21.79

70.55

 

12.69

21.61

52.12

5.86

8.85

27.49

13.48

21.04

66.70

2

100

 

84.75

 

42.85

40.46

84.72

18.48

16.90

31.69

42.61

38.07

85.78

 

22.54

35.81

74.53

11.16

17.18

35.86

25.63

37.95

106.99

5

100

 

92.78

 

67.65

67.89

87.24

21.93

19.78

21.40

67.22

65.94

87.98

 

26.93

57.77

86.46

15.20

18.92

25.35

37.40

75.18

83.91

10

100

 

100.23

 

67.53

68.40

87.99

19.82

20.80

21.38

69.64

68.49

81.88

 

31.36

63.19

88.68

17.39

18.96

21.56

43.26

72.14

84.89