概述
因为最近在做一个es平台的项目，需要熟悉一些es的源码，项目是基于es6.7.0的版本开发的，但是es在高的版本是基于gradle构建的，自己对gradle不熟悉，所以前面一直参考网上的在弄，但是网上和实际的还是有一点差别，所以这里特意记录一下，详细讲解了如何安装es的源码调试环境。
环境准备
win7系统
idea 201802
jdk openjdk-11.0.2
es 6.7.0
gradle-5.2.1
环境搭建步骤
1. 下载es源代码
es源码可以直接从github上面直接clone下来即可。es-github地址: https://github.com/elastic/elasticsearch
这里直接在idea中进行操作，下载完后然后切换到对应的分支，这里是使用的es6.7所以这里切换到此分支。
git checkout -b 6.7 origin/6.7
建议将其fork到自己github仓库，然后从自己的仓库clone到本地。
2. gradle环境准备
gradle的安装这里不做说明，详细的可以参考https://blog.csdn.net/Tomgs/article/details/73865508，这里说明一下如果不知道安装哪个版本的gradle，可以在es的源码目录下面的gradle/wrapper/gradle-wrapper.properties中的distributionUrl属性指定了es版本，所以这里直接根据这个版本进行下载安装即可，es6.7默认用的是gradle-5.2.1。
安装成功后进行如下配置修改工作方便后续步骤的进行。
- 在项目中指定gradle安装包路径
将下载的gradle-5.2.1-all.zip包放到elasticsearch\gradle\wrapper目录下，
确保和elasticsearch\gradle\wrapper\gradle-wrapper.properties在同级目录，
然后修改elasticsearch\gradle\wrapper\gradle-wrapper.properties配置如下：
distributionUrl=gradle-5.2.1-all.zip
- 修改全局gradle仓库地址
在USER_HOME/.gradle/下面创建新文件init.gradle（没有这个文件的可以手动创建），输入下面的内容并保存。
修改gradle的远程仓库地址为阿里云的仓库：
allprojects {
repositories {
def REPOSITORY_URL = 'http://maven.aliyun.com/nexus/content/groups/public/'
all {
ArtifactRepository repo ->
if (repo instanceof MavenArtifactRepository) {
def url = repo.url.toString()
if (url.startsWith('https://repo.maven.org/maven2') || url.startsWith('https://jcenter.bintray.com/')) {
project.logger.lifecycle "Repository ${repo.url} replaced by $REPOSITORY_URL."
remove repo
}
}
}
maven {
url REPOSITORY_URL
}
}
}
其中USER_HOME/.gradle/是自己的gradle安装目录，示例值：C:\Users\Administrator\.gradle，如果没有.gradle目录，可用自己创建
3. 编译es源码
在准备好上述环境之后，下面就可以进行相关的编译工作了，在idea的Terminal运行命令：
./gradlew idea
等待几分钟，这个过程比较慢，如果成功将会看到BUILD SUCCESSFUL的字样。
4. 打包es源码
在编译好后进行对源码进行打包操作，命令如下：
./gradlew -p distribution/archives/tar assemble --parallel
此命令将es源码打包成一个tar形式的压缩包elasticsearch-6.7.0-SNAPSHOT.jar，位于distribution/archives/tar目录下面，为什么这里还需要进行打包，因为es启动的时候需要加载module，如果使用下载的安装包里面的module的方式可能和下载的源码是不一致的导致不能启动成功，这里在后面的遇到的问题章节进行说明。
5. 开始调试es源码
将上面的安装包解压到一个具体的位置如：E:\elasticsearch-6.7.0-SNAPSHOT。
找到es的启动类org.elasticsearch.bootstrap.Elasticsearch，然后运行，正常来说运行会出现错误的，这些遇到的错误在后面的遇到的问题章节进行说明，这里直接把正常的配置贴出来。
在启动的时候添加如下的jvm参数
-Xms512m
-Xmx512m
-Des.path.home=E:\elasticsearch-6.7.0-SNAPSHOT
-Des.path.conf=E:\elasticsearch-6.7.0-SNAPSHOT\config
-Dlog4j2.disable.jmx=true
-Djava.security.policy=E:\elasticsearch-6.7.0-SNAPSHOT\config\java.policy
上面java.policy文件的内容如下：
// default permissions granted to all domains
grant {
permission java.lang.RuntimePermission "createClassLoader";
// allows anyone to listen on dynamic ports
permission java.net.SocketPermission "localhost:0", "listen";
// "standard" properies that can be read by anyone
permission java.util.PropertyPermission "java.version", "read";
permission java.util.PropertyPermission "java.vendor", "read";
permission java.util.PropertyPermission "java.vendor.url", "read";
permission java.util.PropertyPermission "java.class.version", "read";
permission java.util.PropertyPermission "os.name", "read";
permission java.util.PropertyPermission "os.version", "read";
permission java.util.PropertyPermission "os.arch", "read";
permission java.util.PropertyPermission "file.separator", "read";
permission java.util.PropertyPermission "path.separator", "read";
permission java.util.PropertyPermission "line.separator", "read";
permission java.util.PropertyPermission
"java.specification.version", "read";
permission java.util.PropertyPermission "java.specification.vendor", "read";
permission java.util.PropertyPermission "java.specification.name", "read";
permission java.util.PropertyPermission
"java.vm.specification.version", "read";
permission java.util.PropertyPermission
"java.vm.specification.vendor", "read";
permission java.util.PropertyPermission
"java.vm.specification.name", "read";
permission java.util.PropertyPermission "java.vm.version", "read";
permission java.util.PropertyPermission "java.vm.vendor", "read";
permission java.util.PropertyPermission "java.vm.name", "read";
};
然后再启动，会发现启动成功没有报错，这时访问http://localhost:9200/，出现如下结果，说明环境搭建成功了。
{
"name" : "Yqy-_KK",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "heVTcFkoRUihY-hANO3UnA",
"version" : {
"number" : "6.7.0",
"build_flavor" : "unknown",
"build_type" : "unknown",
"build_hash" : "Unknown",
"build_date" : "Unknown",
"build_snapshot" : true,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
上面为什么需要添加这些参数，可以通过报错的信息，来判断缺少什么参数，当然也可以在es的安装包中通过查看es的启动脚本
elasticsearch的内容来确定需要那些参数。exec \ "$JAVA" \ $ES_JAVA_OPTS \ -Des.path.home="$ES_HOME" \ -Des.path.conf="$ES_PATH_CONF" \ -Des.distribution.flavor="$ES_DISTRIBUTION_FLAVOR" \ -Des.distribution.type="$ES_DISTRIBUTION_TYPE" \ -cp "$ES_CLASSPATH" \ org.elasticsearch.bootstrap.Elasticsearch \ "$@"
遇到的问题
- 编译出现
The specified initialization script 'C:\Users\***\AppData\Local\Temp\ijinit1.gradle' does not exist.
这个只需要将idea的缓存清空一下再运行编译命令，
File -> Invalidate Caches/Restart。
- 启动出现
java.lang.IllegalStateException: path.home is not configured
配置jvm参数：
-Des.path.home=E:\elasticsearch-6.7.0-SNAPSHOT
- 启动出现
ERROR: the system property [es.path.conf] must be set
配置jvm参数：
-Des.path.home=E:\elasticsearch-6.7.0-SNAPSHOT\config
- 启动出现
ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
具体堆栈如下：
2019-04-20 18:49:21,809 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:358)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1805)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:318)
at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.logging.log4j.core.jmx.Server.register(Server.java:393)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:168)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:141)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:558)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:263)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:207)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:220)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:197)
at org.elasticsearch.common.logging.LogConfigurator.configureStatusLogger(LogConfigurator.java:250)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:166)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:127)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:302)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
解决办法：添加jvm参数
-Dlog4j2.disable.jmx=true
- 启动出现
java.lang.NoClassDefFoundError: org/elasticsearch/plugins/ExtendedPluginsClassLoader错误
[2019-04-20T18:51:48,279][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] fatal error in thread [main], exiting
java.lang.NoClassDefFoundError: org/elasticsearch/plugins/ExtendedPluginsClassLoader
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:545) ~[main/:?]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[main/:?]
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[main/:?]
at org.elasticsearch.node.Node.<init>(Node.java:339) ~[main/:?]
at org.elasticsearch.node.Node.<init>(Node.java:266) ~[main/:?]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[main/:?]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[main/:?]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[main/:?]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[main/:?]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[main/:?]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[main/:?]
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.plugins.ExtendedPluginsClassLoader
at jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) ~[?:?]
at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[?:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:521) ~[?:?]
... 15 more
解决办法：在idea中的
Run/Debug Configuration页面勾选'Include dependencies with "Provided" scope'
Provided
- 启动出现
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "createClassLoader")错误
[2019-04-22T19:29:52,650][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "createClassLoader")
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[main/:?]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[main/:?]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[main/:?]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[main/:?]
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "createClassLoader")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
at java.security.AccessController.checkPermission(AccessController.java:895) ~[?:?]
at java.lang.SecurityManager.checkPermission(SecurityManager.java:322) ~[?:?]
at java.lang.SecurityManager.checkCreateClassLoader(SecurityManager.java:384) ~[?:?]
at java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:369) ~[?:?]
at java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:359) ~[?:?]
at java.lang.ClassLoader.<init>(ClassLoader.java:456) ~[?:?]
at org.elasticsearch.plugins.ExtendedPluginsClassLoader.<init>(ExtendedPluginsClassLoader.java:36) ~[main/:?]
at org.elasticsearch.plugins.ExtendedPluginsClassLoader.lambda$create$0(ExtendedPluginsClassLoader.java:57) ~[main/:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at org.elasticsearch.plugins.ExtendedPluginsClassLoader.create(ExtendedPluginsClassLoader.java:56) ~[main/:?]
at org.elasticsearch.plugins.PluginLoaderIndirection.createLoader(PluginLoaderIndirection.java:31) ~[main/:?]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:545) ~[main/:?]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[main/:?]
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[main/:?]
at org.elasticsearch.node.Node.<init>(Node.java:339) ~[main/:?]
at org.elasticsearch.node.Node.<init>(Node.java:266) ~[main/:?]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[main/:?]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[main/:?]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[main/:?]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[main/:?]
... 6 more
- 第一种：
在jdk的home目录下面%JAVA_HOME%\conf\security下面找到java.policy，然后打开将如下内容添加到grant里面。
permission java.lang.RuntimePermission "createClassLoader";- 第二种：
在es的config目录下面自己创建一个java.policy文件，然后添加如下内容：grant { permission java.lang.RuntimePermission "createClassLoader"; }然后启动es时指定jvm参数：
-Djava.security.policy=E:\elasticsearch-6.7.0\config\java.policy
- 启动出现
java.lang.NoSuchFieldError: INDEX_SOFT_DELETES_RETENTION_LEASE_PERIOD_SETTING错误
[2019-04-22T19:59:54,021][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] fatal error in thread [main], exiting
java.lang.NoSuchFieldError: INDEX_SOFT_DELETES_RETENTION_LEASE_PERIOD_SETTING
at org.elasticsearch.xpack.ccr.action.TransportResumeFollowAction.<clinit>(TransportResumeFollowAction.java:390) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:49) ~[main/:?]
... 6 more
这个的原因是下载的发行版的安装包和你git下来的代码不一致导致的，因为在指定es的home路径时会从种加载module模块相关的jar包，所以建议是用源码的方式编译打包然后指定到自己打包的环境，虽然说是安装的版本和源码版本是一样的但是实际结果就是不一样，尴尬。
这里也是看到网上很多博客都是下载的和源码版本一样的安装包，就不自己打包了，这里自己打包和源码包对比一下就会发现还是不一样的。这个自己可以把安装包的对应的类的jar包和自己打包的jar包对比一下就知道了，这也是为什么我参考网上一些作者的资料没有运行成功的原因。
- 启动出现：
initial heap size [201326592] not equal to maximum heap size [3202351104]; this can cause resize pauses and prevents mlockall from locking the entire heap
错误如下：
[2019-04-23T11:16:07,203][INFO ][o.e.b.BootstrapChecks ] [node-1] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: initial heap size [201326592] not equal to maximum heap size [3202351104]; this can cause resize pauses and prevents mlockall from locking the entire heap
这个错误的原因是因为我们启动的时候是直接用的idea命令运行的所以启动的jvm堆的参数都是使用的默认的，所以需要我们自己指定或者指定es的jvm.options文件路径（这个相对麻烦）这里就直接手动指定。
-Xms512m -Xmx512m
这些就是我在搭建es的源码编译环境所遇到的问题，这里作一下记录。
参考文章
教你编译调试Elasticsearch 6.3.2源码
渣渣菜鸡的 ElasticSearch 源码解析 —— 环境搭建
