当前位置:首页 > 开发 > 系统架构 > 架构 > 正文

hbase定时memflush PeriodicMemstoreFlusher

发表于: 2015-03-13   作者:blackproof   来源:转载   浏览次数:
摘要: PeriodicMemstoreFlusher类 定时判断是否触发memflush, 判断间隔hbase.server.thread.wakefrequency 默认10S @Override protected void chore() { for (HRegion r : this.server.onlineRegions.values()) {

PeriodicMemstoreFlusher类 定时判断是否触发memflush,

判断间隔hbase.server.thread.wakefrequency 默认10S

   @Override
    protected void chore() {
      for (HRegion r : this.server.onlineRegions.values()) {
        if (r == null)
          continue;
        if (r.shouldFlush()) {
          FlushRequester requester = server.getFlushRequester();
          if (requester != null) {
            long randomDelay = rand.nextInt(RANGE_OF_DELAY) + MIN_DELAY_TIME;
            LOG.info(getName() + " requesting flush for region " + r.getRegionNameAsString() +
                " after a delay of " + randomDelay);
            //Throttle the flushes by putting a delay. If we don't throttle, and there
            //is a balanced write-load on the regions in a table, we might end up
            //overwhelming the filesystem with too many flushes at once.
            requester.requestDelayedFlush(r, randomDelay);
          }
        }
      }
    }

 

触发条件should Flush方法

  boolean shouldFlush() {
    if(this.completeSequenceId + this.flushPerChanges < this.sequenceId.get()) {//防止memstore变化太多
      return true;
    }
    if (flushCheckInterval <= 0) { //disabled
      return false;
    }
    long now = EnvironmentEdgeManager.currentTimeMillis();
    //if we flushed in the recent past, we don't need to do again now
    if ((now - getLastFlushTime() < flushCheckInterval)) {//每次flush间隔时间,hbase.regionserver.optionalcacheflushinterval默认为1小时
      return false;
    }
    //since we didn't flush in the recent past, flush now if certain conditions
    //are met. Return true on first such memstore hit.
    for (Store s : this.getStores().values()) {
      if (s.timeOfOldestEdit() < now - flushCheckInterval) {   //  oldest edit of store  , one hour ago ,  now  有store在一小时钱修改过
        // we have an old enough edit in the memstore, flush
        return true;
      }
    }
    return false;
  }
  

 

  

  1.依照sequeceid,判断memstore没有太多flush,进行flush

  2.比较每次flush的间隔时间,没到时间,不进行flush(单从时间上看,memstore够老)

  3.超过间隔时间,并且有个store的edit超过间隔时间(从用户修改上看,memstore够老)

 

 

 

 

 

 

 

hbase定时memflush PeriodicMemstoreFlusher

  • 0

    开心

    开心

  • 0

    板砖

    板砖

  • 0

    感动

    感动

  • 0

    有用

    有用

  • 0

    疑问

    疑问

  • 0

    难过

    难过

  • 0

    无聊

    无聊

  • 0

    震惊

    震惊

编辑推荐
<img src="http://img.it610.com/image/product/97b20ad29bae4b539b4b30df80fd6a28.jpg" alt="Hb
[转自] http://www.cnblogs.com/flying5/archive/2011/09/15/2178064.html 1. 简介   HBase是一
Hadoop2.5.2 HBase1.0.0 1.安装Hadoop HBase数据存储在HDFS中,因此在安装HBase需要首先安装Hadoop
Hadoop2.5.2 HBase1.0.0 1.安装Hadoop HBase数据存储在HDFS中,因此在安装HBase需要首先安装Hadoop
Hadoop2.5.2 HBase1.0.0 1.安装Hadoop HBase数据存储在HDFS中,因此在安装HBase需要首先安装Hadoop
hbase的文件存储在生产环境下是基于hadoop HDFS文件系统的,HDFS为hbase提供了高容错和分布式的保证
之前我们已经学习过HTable表的定义及基本操作了,本节我们将要学习HBase的高级功能之一过滤器HBase
1. Hadoop生态系统 底层是存储(HDFS),上层是计算框架 从图中可以看出,Hive、Pig和Mahout是基于MapR
1. Hadoop生态系统 底层是存储(HDFS),上层是计算框架 从图中可以看出,Hive、Pig和Mahout是基于MapR
1. Hadoop生态系统 底层是存储(HDFS),上层是计算框架 从图中可以看出,Hive、Pig和Mahout是基于MapR
版权所有 IT知识库 CopyRight © 2009-2015 IT知识库 IT610.com , All Rights Reserved. 京ICP备09083238号