文章开始前先吐槽下Oozie居然要自己编译。官方文档
1.环境准备
Unix box
Java JDK 1.7+(本文用的是1.8)
maven (现在一般是3.3.9)
-
官方文档有要求安装pig,其实没什么卵用。
2.MAVEN配置
maven安装过程这里就省略了。接下来配置maven mirror,可以配置成阿里云的。
<mirror>
<id>nexus</id>
<mirrorOf>*</mirrorOf>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
</mirror>
3.编译
3.1.解压Oozie
tar -xzvf oozie-4.3.0.tar.gz
3.2.修改pom.xml
我的java版本是1.8的,Oozie默认的java版本是1.7
cd oozie-4.3.0
vi pom.xml
<targetJavaVersion>1.7</targetJavaVersion> -> <targetJavaVersion>1.8</targetJavaVersion>
<sourceJavaVersion>1.7</sourceJavaVersion> -> <sourceJavaVersion>1.8</sourceJavaVersion>
3.3.准备缺少的dependency
${mvn_repo}代表本地maven仓库地址
3.3.1 pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
mkdir -p ${mvn_repo}/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde
cd ${mvn_repo}/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde
wget http://repo.spring.io/plugins-release/org/pentaho/pentaho-aggdesigner-\
algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
3.3.2 doxia-module-twiki-1.0-alpha-9.2y.jar
mkdir -p ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y
cd ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y
wget https://repository.cloudera.com/content/repositories/releases/org/\
apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y/doxia-module-twiki-1.0-alpha-9.2y.jar --no-check-certificate
3.3.3 doxia-core-1.0-alpha-9.2y.jar
mkdir -p ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki
cd ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki
wget https://repository.cloudera.com/content/repositories\
/releases/org/apache/maven/doxia/doxia-core/1.0-alpha-9.2y/doxia-core-1.0-alpha-9.2y.jar --no-check-certificate
3.4.编译Oozie
cd oozie-4.3.0
bin/mkdistro.sh -DskipTests -Puber -Phadoop-2 \
-Dhadoop.version=2.7.2 \
-Dhadoop.auth.version=2.7.2 \
-Ddistcp.version=2.7.2 \
-Dsqoop.version=1.4.3 \
-Dhive.version=1.2.0 \
-Dhbase.version=0.94.27
hadoop version需要改成自己安装hadoop version,其他的也是,没安装也没关系。
经过漫长的等待。。。
[INFO] Apache Oozie Main .................................. SUCCESS [ 0.601 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [ 1.371 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [ 0.061 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [ 0.191 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [ 0.011 s]
[INFO] Apache Oozie Client ................................ SUCCESS [ 8.851 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [ 1.710 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [ 1.460 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [ 0.443 s]
[INFO] Apache Oozie Core .................................. SUCCESS [ 29.701 s]
[INFO] Apache Oozie Share Lib Streaming ................... SUCCESS [ 2.492 s]
[INFO] Apache Oozie Share Lib Pig ......................... SUCCESS [ 2.425 s]
[INFO] Apache Oozie Share Lib Hive ........................ SUCCESS [ 3.291 s]
[INFO] Apache Oozie Share Lib Hive 2 ...................... SUCCESS [ 2.734 s]
[INFO] Apache Oozie Share Lib Sqoop ....................... SUCCESS [ 1.063 s]
[INFO] Apache Oozie Examples .............................. SUCCESS [ 2.499 s]
[INFO] Apache Oozie Share Lib Spark ....................... SUCCESS [ 3.877 s]
[INFO] Apache Oozie Share Lib ............................. SUCCESS [ 12.781 s]
[INFO] Apache Oozie Docs .................................. SUCCESS [ 5.929 s]
[INFO] Apache Oozie WebApp ................................ SUCCESS [ 23.219 s]
[INFO] Apache Oozie Tools ................................. SUCCESS [ 1.896 s]
[INFO] Apache Oozie MiniOozie ............................. SUCCESS [ 0.693 s]
[INFO] Apache Oozie Distro ................................ SUCCESS [ 17.542 s]
[INFO] Apache Oozie ZooKeeper Security Tests .............. SUCCESS [ 1.894 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
编译好的包在 oozie-4.3.0/distro/target/oozie-4.3.0-distro.tar.gz
3.5.可能出现的错误
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (default) on project\
oozie-distro: An Ant BuildException has occured: java.net.ConnectException: 连接超时 (Connection timed out) -> [Help]
这里需要下载tomcat,可能因为公司的网有代理导致java连接超时,那就手动下载一下吧。。。
cd oozie-4.3.0/distro/downloads
wget http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.47/bin/apache-tomcat-6.0.47.tar.gz
mv apache-tomcat-6.0.47.tar.gz tomcat-6.0.47.tar.gz
需要重命名下文件名,然后修改oozie-distro.xml,删了下面三行代码,然后再编译一次吧。
vi oozie-4.3.0/distro/oozie-distro.xml
<!--<mkdir dir="downloads"/>
<get src="http://archive.apache.org/dist/tomcat/tomcat-6/v${tomcat.version}/bin/apache-tomcat-${tomcat.version}.tar.gz"
dest="downloads/tomcat-${tomcat.version}.tar.gz" verbose="true" skipexisting="true"/>-->