https://www.myziyuan.com/
- rushou
- 把你编译后的hadoop源码丢到原来的hadoop集群环境中去 即覆盖hadoop安装目录下的原hadoop-core-xxx.jar 同样的所有节点都需要更新 然后重启集群
- 2021-02-26 16:50:01
- 515680997
- 1.编译hadoop2.6.0的eclipse插件编译源码：cd src/contrib/eclipse-plugin ant jar -Dversion=2.6.0 -Declipse.home=/opt/eclipse -Dhadoop.home=/opt/hadoop-2.6.0 eclipse.home 和 hadoop.home 设置成你自己的环境路径命令行执行编译，产生了8个警告信息，直接忽略。compile: [echo] contrib: eclipse-plugin [javac] /software/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin/build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds [javac] Compiling 45 source files to /software/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin/classes [javac] /opt/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar(org/apache/hadoop/fs/Path.class): warning: Cannot find annotation method 'value()' in type 'LimitedPrivate': class file for org.apache.hadoop.classification.InterfaceAudience not found [javac] /opt/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-
- 2021-02-12 03:40:34
- 互站网
- 如何在hadoop,在不使用eclipse情况使java程序在hadoop 2.2中运行的完整过程。整个过程中其实分为java程序的编译，生成jar包，运行测试。这三个步骤运用的命令都比较简单，主要的还是如何找到hadoop 2.2提供给java程序用来编译的jar包。具体可以查看:HADOOP_HOME/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib目录下面会通过一个在hadoop中创建一个目录的JAVA例子来进行演示具体代码如下：package com.wan.demo;import java.io.IOException;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.FileSystem;import org.apache.hadoop.fs.Path;public class HADemo {public static void main(String[] args) {// TODO Auto-generated method stubmkdir(args[0]);}public static void mkdir(String dir){Configuration configuration=new Configuration();FileSystem fs;try {fs = FileSystem.get(configuration);fs.mkdirs(new Path(dir));fs.close();} catch (IOException e) {// TODO Auto-generated catch blocke.printStackTrace();}}}把HADemo.java文件拷贝到linux环境中配置HADOOP_HOME/bin到环境中，启动集群，进入HADemo.java文件目录中注:下面的lib目录里面的文件由HADOOP_HOME/share/hadoop/httpfs/tomcat/webapps/ webhdfs/WEB-INF/lib目录中获取，下面做的目的是为了缩减命令长度1.编译java# mkdir class#Javac -classpath .:lib/hadoop-common-2.2.0.jar:lib/hadoop-annotations-2.2.0.jar -d class HADemo.java2.生成jar包#jar -cvf hademo.jar -C class/ .added manifestadding: com/(in = 0) (out= 0)(stored 0%)adding: com/wan/(in = 0) (out= 0)(stored 0%)adding: com/wan/demo/(in = 0) (out= 0)(stored 0%)adding: com/wan/demo/HADemo.class(in = 844) (out= 520)(deflated 38%)3.测试运行#hadoop jar hademo.jar com.wan.demo.HADemo /test检测：#hadoop fs -ls /结束！
- 2021-02-12 03:40:34
开通会员