当前位置:首页 > 开发 > 系统架构 > 架构 > 正文

kafka replication tools

发表于: 2015-07-01   作者:blackproof   来源:转载   浏览次数:
摘要: kafka工具   replication tool工作步骤(异步过程,执行完1就结束): 1.更新zk上的/admin/preferred_replica_election节点,内容为副本的几个位置(不包含已经奔溃的leader)   2.controller中的zk的listener读取topic partition的副本的几个位置   3.cont

kafka工具

 

replication tool工作步骤(异步过程,执行完1就结束):

1.更新zk上的/admin/preferred_replica_election节点,内容为副本的几个位置(不包含已经奔溃的leader)

 

2.controller中的zk的listener读取topic partition的副本的几个位置

 

3.controller获取每个topic partition的副本的几个replica中的第一个,作为referred replica;

  如果这个preferred replica不是leader且在isr中,则controller发出request到这个preferred replica的broker,通知其作为topic partition的leader

  

  

bin/kafka-preferred-replica-election.sh --zookeeper localhost:81821/kafka-zk --path-to-json-file topicPartitionList.json

 

topicPartitionList.json格式为:

{

 "partitions":

  [

    {"topic": "topic1", "partition": "0"},

    {"topic": "topic1", "partition": "1"},

    {"topic": "topic1", "partition": "2"},

 

    {"topic": "topic2", "partition": "0"},

    {"topic": "topic2", "partition": "1"},

  ]

}

 

FAQ

What happens if the preferred replica is not in the ISR?

The controller will fail to move the leadership to the preferred replica if it is not in the ISR. This is to ensure that there is no dataloss. When the replica becomes "in-sync" with the leader, the tool can be run again to move the leader.

则移动失败,避免数据丢失

 

How to find if all the partitions have been moved to the "preferred replica" after running the tool?

ListTopicCommand is an excellent tool that provides an overview of all the topic partitions in the cluster. For each topic partition, it displays the leader, assigned replicas and current "in-sync" replica set. If the leader and the first replica in the assigned replica set are the same then the Preferred replica leader election" tool succeeded. If not, the tool failed and may have to be run again.

跟isr的第一个replica相同

kafka replication tools

  • 0

    开心

    开心

  • 0

    板砖

    板砖

  • 0

    感动

    感动

  • 0

    有用

    有用

  • 0

    疑问

    疑问

  • 0

    难过

    难过

  • 0

    无聊

    无聊

  • 0

    震惊

    震惊

编辑推荐
1. kafka原理 1-1 基本构成 1-2 工作原理 ① 每个partition会创建3个备份replica,并分配到broker集
kafka自带了很多工具类,在源码kafka.tools里可以看到: 这些类该如何使用呢,kafka的设计者早就为
http://www.infoq.com/cn/articles/kafka-analysis-part-3?utm_source=infoq&utm_medium=related_co
http://www.infoq.com/cn/articles/kafka-analysis-part-2?utm_source=infoq&utm_medium=related_co
本文已授权InfoQ独家发表,如需转载请注明出处并与InfoQ中文站联系。InfoQ首发地址为 http://www.in
Kafka在0.8以前的版本中,并不提供High Availablity机制,一旦一个或多个Broker宕机,则宕机期间其
本文已授权InfoQ独家发表,如需转载请注明出处并与InfoQ中文站联系。InfoQ首发地址为http://www.inf
Kafka是由LinkedIn开发的一个分布式的消息系统,使用Scala编写,它以可水平扩展和高吞吐率而被广泛
1.概述   在《Kafka实战-入门》一篇中,为大家介绍了Kafka的相关背景、原理架构以及一些关键知识
【原创声明】本文属作者原创,已授权InfoQ中文站首发,转载请务必在文章开头标明出自“Jason's Blog
版权所有 IT知识库 CopyRight © 2009-2015 IT知识库 IT610.com , All Rights Reserved. 京ICP备09083238号