apache atlas项目搭建（1.0.0，2.0.0），centos7环境

一、前置条件

1、jdk1.8、maven3、nodejs

jdk、maven解压、配置环境变量；

nodejs安装，参考 https://www.cnblogs.com/jizhong/p/12102100.html

centos7自带python2.7.5，无需安装

2、配置环境变量 /etc/profile

JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export export PATH=${JAVA_HOME}/bin:${PATH}
MAVEN_HOME=/usr/local/tools/apache-maven-3.6.3
export PATH=${MAVEN_HOME}/bin:${PATH}
NODE_HOME=/usr/local/tools/node
export PATH=${NODE_HOME}/bin:${PATH}

配置生效 source /etc/profile

3、查看版本信息：

node -v

v6.12.3

npm -v

3.10.10

python -V

Python 2.7.5

java -version

mvn –v

4、使用淘宝的npm镜像cnpm

npm install -g cnpm --registry=https://registry.npm.taobao.org

5、maven 配置文件（阿里镜像、仓库地址、jdk版本）

替换maven源，阿里云的源

打开maven配置文件 settings.xml

<mirror>
    <id>alimaven</id>
    <name>aliyun maven</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
    <mirrorOf>central</mirrorOf>
</mirror>

指定下载资源位置

<localRepository>/usr/local/maven/repository</localRepository>

指定jdk版本（本座没有配置）

<profile>    
    <id>jdk-1.8</id>    
    <activation>    
        <activeByDefault>true</activeByDefault>    
        <jdk>1.8</jdk>    
    </activation>    
    <properties>    
        <maven.compiler.source>1.8</maven.compiler.source>    
        <maven.compiler.target>1.8</maven.compiler.target>    
        <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>    
    </properties>
</profile>

二、apache atlas1.0.0

1、下载源系统 http://atlas.apache.org/#/Downloads

企业微信截图_15882361078763.png

2、编译

tar -zxvf apache-atlas-1.0.0-sources.tar.gz cd apache-atlas-sources-1.0.0

配置堆大小，防止内存溢出

export MAVEN_OPTS="-Xms2g -Xmx4g"

编译，两种方法选其一

1）、该方式编译不会内嵌HBase和Solr

mvn clean -DskipTests package -Pdist

2）、使用atlas内嵌HBase和Solr，测试用这种方式，注意：embedded-hbase-solr前面没有空格

mvn clean -DskipTests package -Pdist,embedded-hbase-solr

编译完成后的jar包位于目录：\apache-atlas-sources-1.1.0\distro\target 中，压缩包有对应的解压文件，可直接部署；或者自己加压到指定路径

3、部署，进入 apache-atlas-1.0.0 路径

/usr/local/projects/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0

用内嵌的 hbase 和 solr 启动，配置hbase和solr跟随atlas启动和停止

export MANAGE_LOCAL_HBASE=true

export MANAGE_LOCAL_SOLR=true

上面的两步已经在 atlas-env.sh 文件默认配置

4、启动

bin/atlas_start.py

5、查看进程 jps

企业微信截图_158823674822.png

6、添加自带例子 bin/quick_start.py

Enter username for atlas :- admin Enter password for atlas :- admin

7、管理界面登录入口 http://localhost:21000 用户名：admin 密码：admin

8、停止atlas，bin/atlas_stop.py

在hbase里配置了JAVE_HOME

三、apache atlas2.0.0

编译过程和1.0.0一致，编译时间较1.0更长

编译完成后使用

/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-server/apache-atlas-2.0.0

1、修改配置文件 apache-atlas-2.0.0-server/apache-atlas-2.0.0/conf/

vim atlas-env.sh
export JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export MANAGE_LOCAL_HBASE=true  #如果要使用外部的zk和hbase，则改为false
export MANAGE_LOCAL_SOLR=true  #如果要是用外部的solr，则改为false

vim atlas-application.properties
# Hbase地址（对应的zk地址）配置（自带hbase会根据此端口启动一个zk实例）
atlas.graph.storage.hostname=localhost:2181 # 如果使用外部hbase，则填写外部zookeeper地址

# Solr地址配置
atlas.graph.index.search.solr.http-urls=http://localhost:8984/solr

# Kafka相关配置
atlas.notification.embedded=true # 如果要使用外部的kafka，则改为false
# 内嵌kafka会根据此端口启动一个zk实例
atlas.kafka.zookeeper.connect=localhost:9026 # 如果使用外部kafka，则填写外部zookeeper地址
atlas.kafka.bootstrap.servers=localhost:9027 # 如果使用外部kafka，则填写外部broker server地址

2、手动启动hbase

# 进入atlas自带的hbase目录
apache-atlas-2.0.0-server/apache-atlas-2.0.0/hbase 
# 启动 hbase 
bin/start-hbase.sh

3、手动启动solr，为solr创建初始化index库

# 进入atlas自带的solr目录，
apache-atlas-2.0.0-server/apache-atlas-2.0.0/solr
# 启动solr
bin/solr start -c -z localhost:2181 -p 8984 -force

# 创建初始化collections
bin/solr create -c vertex_index -shards 1 -replicationFactor 1 -force
bin/solr create -c edge_index -shards 1 -replicationFactor 1 -force
bin/solr create -c fulltext_index -shards 1 -replicationFactor 1 -force

然后打开浏览器访问solr的web服务如下，则solr启动成功 http://localhost:8984/

4、启动atlas，apache-atlas-2.0.0-server/apache-atlas-2.0.0

bin/atlas_start.py

最后添加自带例子，同1.0.0

四、访问路径

atlas：http://192.168.198.129:21000/

solr(1.0.0)：http://192.168.65.139:9838/

solr(2.0.0)：http://localhost:8984/

五、问题

1、

[ERROR] Java heap space -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError

解决：

没有设置 MAVEN_OPTS="-Xms2g -Xmx4g"，造成堆内存溢出

2、

[ERROR] [ERROR] Some problems were encountered while processing the POMs:

[FATAL] Non-resolvable parent POM for org.apache.atlas:apache-atlas:1.0.0: Could not transfer artifact org.apache:apache:pom:17 from/to alimaven (http://maven.aliyun.com/nexus/content/groups/public/): Transfer failed for http://maven.aliyun.com/nexus/content/groups/public/org/apache/apache/17/apache-17.pom and 'parent.relativePath' points at wrong local POM @ line 23, column 13

编译错误：

晚上下班突然就可以了

3、

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-npm (install node and npm) on project atlas-dashboardv2: Could not extract the Node archive: Could not extract archive: '/usr/local/maven/repository/com/github/eirslett/node/4.4.2/node-4.4.2-linux-x64.tar.gz': EOFException -> [Help 1]

Failed to run task: 'npm install' failed

npm: command not found...

需要安装nodejs，并使用淘宝的npm镜像cnpm

npm install -g cnpm --registry=https://registry.npm.taobao.org

4、

Creating sample types:

Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@491b9b8 failed with status 503 (Service Unavailable) Response Body (<html>

<head>

<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>

<title>Error 503 </title>

</head>

<body>

<h2>HTTP ERROR: 503</h2>

<p>Problem accessing /api/atlas/v2/types/typedefs/. Reason:

<pre> Service Unavailable</pre></p>

<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>

</body>

</html>

)

at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:395)

at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:323)

at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:211)

at org.apache.atlas.AtlasClientV2.createAtlasTypeDefs(AtlasClientV2.java:227)

at org.apache.atlas.examples.QuickStartV2.createTypes(QuickStartV2.java:185)

at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141)

at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:126)

No sample data added to Apache Atlas Server.

curl -v -u username:password http://localhost:21000/api/atlas/admin/version

解决：

solr启动失败，还是配置的问题

5、

hbase Error: JAVA_HOME is not set

解决：

/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hbase/conf

atlas 1.0.0 配置了JAVA_HOME

hbase-env.sh

6、

cannot create temp file for here-document: No space left on device

空间满了 df -h

7、

Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at localhost:2181: cluster not found/not ready

at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385)

at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141)

at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383)

at org.janusgraph.diskstorage.solr.Solr6Index.<init>(Solr6Index.java:218)

2181 zookeeper 默认端口

还是配置的问题，atlas2.0.0