当前位置:首页 > 开发 > 开源软件 > 正文

【Hadoop三】Centos7_x86_64部署Hadoop集群之编译Hadoop源代码

发表于: 2014-12-02   作者:bit1129   来源:转载   浏览:
摘要:   编译必需的软件 Firebugs3.0.0 Maven3.2.3 Ant JDK1.7.0_67 protobuf-2.5.0 Hadoop 2.5.2源码包       Firebugs3.0.0   http://sourceforge.jp/projects/sfnet_findbug
 

编译必需的软件

  • Firebugs3.0.0
  • Maven3.2.3
  • Ant
  • JDK1.7.0_67
  • protobuf-2.5.0
  • Hadoop 2.5.2源码包

 

 

 

  • Firebugs3.0.0

 

http://sourceforge.jp/projects/sfnet_findbugs/downloads/findbugs/3.0.0/findbugs-3.0.0.tar.gz/
tar xzvf findbugs-3.0.0
export FINDBUGS_HOME=/home/hadoop/software/findbugs-3.0.0
export PATH=$FINDBUGS_HOME/bin:$PATH

 

第一次构件时没有安装FireBugs,结果报如下错误

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project hadoop-common: An Ant BuildException has occured: stylesheet /home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/${env.FINDBUGS_HOME}/src/xsl/default.xsl doesn't exist.
[ERROR] around Ant part ...<xslt style="${env.FINDBUGS_HOME}/src/xsl/default.xsl" in="/home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/target/findbugsXml.xml" out="/home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/target/site/findbugs.html"/>... @ 44:267 in /home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]

 

 

  • Maven3.2.3

 

export M2_HOME=/home/hadoop/software/apache-maven-3.2.3
export PATH=$M2_HOME/bin:$PATH

 

  • Ant
export ANT_HOME=/home/hadoop/software/apache-ant-1.9.4
export PATH=$ANT_HOME/bin:$PATH

 

  • JDK1.7.0_67

 

export JAVA_HOME=/home/hadoop/software/jdk1.7.0_67
export PATH=$JAVA_HOME/bin:$PATH

 

  • protobuf-2.5.0

 

tar zxvf protobuf-2.5.0.tar.gz  
cd protobuf-2.5.0  
./configure --prefix=/usr/local  
make 
make install 

   

  • Hadoop 2.5.2源码包
tar zxvf hadoop-2.5.2-src.tar.gz  
cd hadoop-2.5.2-src  
mvn package -Pdist,native,docs -DskipTests -Dtar

  • 编译结果

Maven构建大概持续1个小时,构建的结果应该显示所有的构建都正确,如下所示:

 

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop Main ................................. SUCCESS [  3.460 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  2.440 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  8.027 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.502 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  6.744 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  7.694 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 10.779 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [01:08 min]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  3.241 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [07:05 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 11.771 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.039 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [14:51 min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [05:01 min]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [01:45 min]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  5.854 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.049 s]
[INFO] hadoop-yarn ........................................ SUCCESS [  0.066 s]
[INFO] hadoop-yarn-api .................................... SUCCESS [01:24 min]
[INFO] hadoop-yarn-common ................................. SUCCESS [01:32 min]
[INFO] hadoop-yarn-server ................................. SUCCESS [  0.096 s]
[INFO] hadoop-yarn-server-common .......................... SUCCESS [ 45.766 s]
[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [01:45 min]
[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [  4.768 s]
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [  9.864 s]
[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 23.242 s]
[INFO] hadoop-yarn-server-tests ........................... SUCCESS [  0.779 s]
[INFO] hadoop-yarn-client ................................. SUCCESS [  8.364 s]
[INFO] hadoop-yarn-applications ........................... SUCCESS [  0.068 s]
[INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [  4.872 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [  2.523 s]
[INFO] hadoop-yarn-site ................................... SUCCESS [  0.051 s]
[INFO] hadoop-yarn-project ................................ SUCCESS [  9.001 s]
[INFO] hadoop-mapreduce-client ............................ SUCCESS [  0.124 s]
[INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 35.116 s]
[INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 27.584 s]
[INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [  7.655 s]
[INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 17.608 s]
[INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 15.425 s]
[INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [ 39.538 s]
[INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [  2.738 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  9.863 s]
[INFO] hadoop-mapreduce ................................... SUCCESS [  4.919 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 19.115 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 12.291 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [  3.078 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [ 11.125 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  7.794 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [  6.021 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [  5.012 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 10.999 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  8.789 s]
[INFO] Apache Hadoop Client ............................... SUCCESS [ 10.152 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  0.165 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 18.100 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [  6.974 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.028 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [02:37 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44:48 min
[INFO] Finished at: 2014-12-03T08:12:48-05:00
[INFO] Final Memory: 144M/483M
[INFO] ------------------------------------------------------------------------

 

 

编译得到的安装包信息是

/home/hadoop/software/hadoop-2.5.2-src/hadoop-dist/target/hadoop-2.5.2.tar.gz
 

【Hadoop三】Centos7_x86_64部署Hadoop集群之编译Hadoop源代码

  • 0

    开心

    开心

  • 0

    板砖

    板砖

  • 0

    感动

    感动

  • 0

    有用

    有用

  • 0

    疑问

    疑问

  • 0

    难过

    难过

  • 0

    无聊

    无聊

  • 0

    震惊

    震惊

编辑推荐
虚机环境配置 目前只有一台虚拟机,设置IP为192.168.26.131,对这台虚拟机进行环境和Hadoop相关的配
虚机环境配置 目前只有一台虚拟机,设置IP为192.168.26.131,对这台虚拟机进行环境和Hadoop相关的配
虚机环境配置 目前只有一台虚拟机,设置IP为192.168.26.131,对这台虚拟机进行环境和Hadoop相关的配
1. Cloudera介绍 Hadoop是一个开源项目,Cloudera对Hadoop进行了商业化,简化了安装过程,并对hadoo
1. Hadoop集群介绍 Hadoop集群部署,就是以Cluster mode方式进行部署。 Hadoop的节点构成如下: HDF
1、基本工具 yum、scp、curl、wget、pdsh、ssh 2、环境准备 2.1、系统环境 Centos6.5 64bit Ambari
本次hadoop集群部署,利用vmware安装linux系统,并在linux上进行hadoop集群部署测试。 需要用到的软
  Hadoop的集群部署和单节点部署类似,配置文件不同,另外需要修改网络方面的配置   首先,准备
由于Hadoop的MapReduce和HDFS都有通信的需求,需要对通信的对象进行序列化。Hadoop并没有采用Java的
  由于Hadoop的MapReduce和HDFS都有通信的需求,需要对通信的对象进行序列化。Hadoop并没有采用Ja
版权所有 IT知识库 CopyRight © 2009-2015 IT知识库 IT610.com , All Rights Reserved. 京ICP备09083238号