当前位置:首页 > 开发 > 系统架构 > 架构 > 正文

hbase 报java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString

发表于: 2014-12-10   作者:blackproof   来源:转载   浏览次数:
摘要: 问题描述         在HBase上运行MapReduce作业时,报如下异常:IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralBy

问题描述

        在HBase上运行MapReduce作业时,报如下异常:IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString

        使用HBase环境如下:CDH5.0.1, HBase版本:0.96.1

问题原因

        This isssue occurs because of an optimization introduced in HBASE-9867 that inadvertently introduced a classloader dependency. This affects both jobs using the -libjars option and "fat jar," jobs which package their runtime dependencies in a nested lib folder.

        这个问题的发生是由于优化了HBASE-9867 引起的,无意间引进了一个依赖类加载器。它影响使用-libjars参数和使用 fat jar两种模式的job. 

        fat jar模式Hadoop的一个特殊功能:可以读取操作目录中/lib目录下包含的所有库的JAR文件,把运行job依赖的jar放在jar中的lib目录下。

解决方案

    1、永久解决方案

         cd /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop

         ln -s /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar hbase-protocol-0.96.1.1-cdh5.0.0.jar

    2: 临时解决方案

    HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf hadoop  jar myjar.jar MyJobMainClass

详细报错信息

 

14/07/16 14:40:50 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table: 
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions:
Wed Jul 16 14:32:49 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405,java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString
Wed Jul 16 14:32:52 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:32:57 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:33:07 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:33:28 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:34:08 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:35:49 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:37:29 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:39:10 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:40:50 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405,java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString


        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
        at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:705)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:144)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1102)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1162)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1054)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1011)
        at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:326)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:192)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
        at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
        at mapreduce.ImportFromFile.run(ImportFromFile.java:228)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at mapreduce.ImportFromFile.main(ImportFromFile.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
        at mapreduce.Driver.main(Driver.java:29)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:897)
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:131)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1402)
        at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:701)
        at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:699)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
        ... 38 more
Exception in thread "main" java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:897)
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:131)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1402)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1176)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1054)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1011)
        at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:326)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:192)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
        at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
        at mapreduce.ImportFromFile.run(ImportFromFile.java:228)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at mapreduce.ImportFromFile.main(ImportFromFile.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
        at mapreduce.Driver.main(Driver.java:29)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

hbase 报java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString

  • 0

    开心

    开心

  • 0

    板砖

    板砖

  • 0

    感动

    感动

  • 0

    有用

    有用

  • 0

    疑问

    疑问

  • 0

    难过

    难过

  • 0

    无聊

    无聊

  • 0

    震惊

    震惊

编辑推荐
背景 在HMaster、RegionServer内部,创建了RpcServer实例,并与Client三者之间实现了Rpc调用,HBase
陈硕 (giantchen_AT_gmail) Blog.csdn.net/Solstice t.sina.com.cn/giantchen 这篇文章要解决的问题
这篇文章要解决的问题是:在接收到 protobuf 数据之后,如何自动创建具体的 Protobuf Message 对象
这篇文章要解决的问题是:在接收到 protobuf 数据之后,如何自动创建具体的 Protobuf Message 对象
Protobuf-dt是Google针对Eclipse编辑器开发的协议缓冲器插件,提供所有IDE编辑器所具备的功能,包括
Protobuf-dt是Google针对Eclipse编辑器开发的协议缓冲器插件,提供所有IDE编辑器所具备的功能,包括
最近项目抓取的数据,存在Hbase里,所以需要从Hbase里读取数据,创建连接时,发现总是报如下的一个异
最近项目抓取的数据,存在Hbase里,所以需要从Hbase里读取数据,创建连接时,发现总是报如下的一个异
最近项目抓取的数据,存在Hbase里,所以需要从Hbase里读取数据,创建连接时,发现总是报如下的一个异
最近项目抓取的数据,存在Hbase里,所以需要从Hbase里读取数据,创建连接时,发现总是报如下的一个异
版权所有 IT知识库 CopyRight © 2009-2015 IT知识库 IT610.com , All Rights Reserved. 京ICP备09083238号