当前位置:首页 > 开发 > 互联网 > 正文

HBase Backup Options

发表于: 2012-08-23   作者:chenchao051   来源:转载   浏览次数:
摘要: If you are thinking about using HBase you will likely want to understand HBase backup options.  I know we did, so let us share what we found.  Please let us know what we missed and what you
If you are thinking about using HBase you will likely want to understand HBase backup options.  I know we did, so let us share what we found.  Please let us know what we missed and what you use for HBase backup!

Export

You could export your tables using the Export (org.apache.hadoop.hbase.mapreduce.Export) MapReduce job that will export the table data into a Sequence File on HDFS.  This was implemented in HBASE-1684 if you want to check out the patch or comments there.  This tool works on one table at a time, so if you need to backup multiple tables, run this on each table.  The exported data can then be imported back into HBase by the Import tool.

Copy Table

If you have another HBase cluster that you want to treat as a backup cluster, you can use the handy CopyTable tool to copy a table at a time.

Distcp

You could use Hadoop’s distcp command to copy the whole /hbase directory from one HDFS cluster to the other.  However, this can leave your data in an inconsistent state, so it should be avoided.  See http://search-hadoop.com/m/wkMgSjVLDb

At this point we should point out that all of the above backup methods are per-table.  Moreover, they don’t work or create a snapshot of the table.   Export and CopyTable are atomic only at the row level.  Furthermore, if you have multiple tables whose tables depend on each other, if they are being modified while you are exporting or copying them, you will end up with inconsistent data – the data in those tables will not be in sync.  See http://search-hadoop.com/m/Q4bU81G116p.

Backup from Mozilla

Because of the above mentioned issues with distcp when running it over a cluster whose data is being modified while distcp is running, developers at Mozilla came up with their own Backup tool.  They’ve described the tool and its use in the popular Migrating HBase in the Trenches post.

Cluster Replication

HBase has a relatively new and not yet widely used whole cluster replication mechanism.  The backup cluster does not have to be identical to the master cluster, which means that the backup cluster could be much less powerful and thus cheaper, while still having enough storage to serve as backup.

Table Snapshot

Ah, the infamous HBASE-50!  This issue saw some great work during GSoC 2010, but it looks like it was never integrated into HBase.  It is unclear whether the contributor simply ran out of steam or time or whether it became apparent that table snapshots are too difficult to implement or simply not doable because of highly distributed nature of HBase.  The JIRA issue does contain patches you can look at, and the author has a now inactive hbase-snapshot repository up on Github.

HDFS Replication

You could also simply crank up the replication factor to the level that makes you feel safe and call that a backup.  This may not guard against data corruption, but it does guard against certain partial hardware failures.

Since so many people seem to be asking about HBase backup options, I hope this serves as a good point-in-time snapshot, a summary of all HBase backup options that are currently on the table.  With time, this will be added to the HBase Book.

Are there other HBase backup options we should have included?

What you use for HBase backup?

原文: http://blog.sematext.com/2011/03/11/hbase-backup-options/
另有一篇博文也可以参考一下: http://blog.csdn.net/jingling_zy/article/details/7554676

HBase Backup Options

  • 0

    开心

    开心

  • 0

    板砖

    板砖

  • 0

    感动

    感动

  • 0

    有用

    有用

  • 0

    疑问

    疑问

  • 0

    难过

    难过

  • 0

    无聊

    无聊

  • 0

    震惊

    震惊

编辑推荐
as some distribute systems ,a spare/second master is always supplied to guarantee high availa
2 HBase
在单机安装Hbase的方法。会引导你通过shell创建一个表,插入一行,然后删除它,最后停止Hbase。只要
3 HBase
HBase 系统架构 HBase是Apache Hadoop的数据库,能够对大型数据提供随机、实时的读写访问。HBase的
4 hbase
hbase hbase HBase是一个分布式的、面向列的开源数据库,该技术来源于 Fay Chang 所撰写的Google论
5 HBASE
HBASE HBase –Hadoop Database,是一个高可靠性、高性能、面向列、可伸缩的分布式存储系统,利用HBs
这个选项是在服务器启用SQL Mail。有两个值,一个是0表示SQL Mail不可用(默认值),另一个为1表示SQ
1. Hadoop生态系统 底层是存储(HDFS),上层是计算框架 从图中可以看出,Hive、Pig和Mahout是基于MapR
1. Zookeeper Dump 访问HBase的web页面:http://192.168.26.140:16030/zk.jsp HBase is rooted at /
1. Hadoop生态系统 底层是存储(HDFS),上层是计算框架 从图中可以看出,Hive、Pig和Mahout是基于MapR
1. Zookeeper Dump 访问HBase的web页面:http://192.168.26.140:16030/zk.jsp HBase is rooted at /
版权所有 IT知识库 CopyRight © 2009-2015 IT知识库 IT610.com , All Rights Reserved. 京ICP备09083238号