当前位置:首页 > 开发 > 开源软件 > 正文

【Kafka十四】关于auto.offset.reset[Q/A]

发表于: 2015-04-14   作者:bit1129   来源:转载   浏览:
摘要: I got serveral questions about  auto.offset.reset. This configuration parameter governs how  consumer read the message from  Kafka when  there is no initial offset in ZooKeeper or
I got serveral questions about  auto.offset.reset. This configuration parameter governs how  consumer read the message from  Kafka when  there is no initial offset in ZooKeeper or if an offset is out of range. 
Q1. "no initial offset in zookeeper "  means that there isn't any consumer to consume the message yet(The offset is set once the consumer starts to consume)?
-- Yes, or if you consumed messages, but auto offset commit is disabled and you haven't explicitly committed any offsets.
Q2:   What does "offset is out of range" mean? Can you eleborate one scenario when  "offset is out of range" could happen?
Kafka uses a retention policy for topics to expire data and clean it up. If some messages expire and your consumer hasn't run in a while, the last committed offset may no longer exist
 
auto.offset.reset  has two values:smallest and largest.
Assume one scenario: A producer has produced 10 messages to kafka, and there is no consumer yet to consume it.
Q3: If  auto.offset.reset is set to "smallest", does it mean that the consumer will read the message from the offset 0?(0 is smallest here)
YES
Q4: If  auto.offset.reset is set to "largest", does it mean that the consumer will not read any message but wait until new messages come?
Also correct. This is why in the quickstart you need to use the --from-beginning flag on the console consumer. Since the consumer is executed after the console producer it wouldn't see any messages unless it set auto.offset.reset to smallest, which is what --from-beginning does.
 

【Kafka十四】关于auto.offset.reset[Q/A]

  • 0

    开心

    开心

  • 0

    板砖

    板砖

  • 0

    感动

    感动

  • 0

    有用

    有用

  • 0

    疑问

    疑问

  • 0

    难过

    难过

  • 0

    无聊

    无聊

  • 0

    震惊

    震惊

编辑推荐
对Kafka offset的管理,一直没有进行系统的总结,这篇文章对它进行分析。 什么是offset offset是con
对Kafka offset的管理,一直没有进行系统的总结,这篇文章对它进行分析。 什么是offset offset是con
对Kafka offset的管理,一直没有进行系统的总结,这篇文章对它进行分析。 什么是offset offset是con
4 kafka
kafka设计原理介绍 背景介绍 Kafka简介   Kafka是一种分布式的,基于发布/订阅的消息系统。主要设
5 kafka
topics主题 producers 生产者(发布消息到主题) consumers 消费者(订阅主题并处理消息) broker
6 kafka
【原创声明】本文属作者原创,已授权InfoQ中文站首发,转载请务必在文章开头标明出自“Jason's Blog
7 kafka
转载自:http://www.cnblogs.com/tangr206/articles/2274845.html 基本概念 kafka的工作方式和其他M
问题: 如何在Outlook for CRM中,将实体记录的预览窗口的信息做调整? 解决方案: 在Outlook里,在
下面概要讲述一下我在设计完成服务器模块和设计客户端后台中遇到的问题及解决方案。 服务器: 1、服
可以证明,I路和Q路交换后的信号的频谱与原信号的频谱是镜像对称的,具体的证明方法如下: 如图中进
版权所有 IT知识库 CopyRight © 2009-2015 IT知识库 IT610.com , All Rights Reserved. 京ICP备09083238号