1、[phoenix]通过HBase命令行单独启动异步填充索引MR(run on yarn)时报如下错误,需升级hbase/lib下的disruptor-3.3.0.jar到disruptor-3.3.11.jar
Exception in thread "main" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: com.lmax.disruptor.dsl.Disruptor.<init>(Lcom/lmax/disruptor/EventFactory;ILjava/util/concurrent/ThreadFactory;Lcom/lmax/disruptor/dsl/ProducerType;Lcom/lmax/disruptor/WaitStrategy;)V
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2254)
at com.google.common.cache.LocalCache.get(LocalCache.java:3985)
at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4788)
at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:241)
at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.phoenix.mapreduce.util.ConnectionUtil.getConnection(ConnectionUtil.java:98)
at org.apache.phoenix.mapreduce.util.ConnectionUtil.getInputConnection(ConnectionUtil.java:57)
at org.apache.phoenix.mapreduce.util.ConnectionUtil.getInputConnection(ConnectionUtil.java:45)
at org.apache.phoenix.mapreduce.index.IndexTool.run(IndexTool.java:532)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.phoenix.mapreduce.index.IndexTool.main(IndexTool.java:756)
Caused by: java.lang.NoSuchMethodError: com.lmax.disruptor.dsl.Disruptor.<init>(Lcom/lmax/disruptor/EventFactory;ILjava/util/concurrent/ThreadFactory;Lcom/lmax/disruptor/dsl/ProducerType;Lcom/lmax/disruptor/WaitStrategy;)V
at org.apache.phoenix.log.QueryLoggerDisruptor.<init>(QueryLoggerDisruptor.java:72)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.<init>(ConnectionQueryServicesImpl.java:413)
at org.apache.phoenix.jdbc.PhoenixDriver$3.call(PhoenixDriver.java:248)
at org.apache.phoenix.jdbc.PhoenixDriver$3.call(PhoenixDriver.java:241)
at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4791)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3584)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2372)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2335)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2250)
... 14 more
<dependency>
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<version>3.3.11</version>
</dependency>
2、[phoenix]通过HBase命令行单独启动异步填充索引MR(run on yarn)时发现yarn状态处于ACCEPTD
,但是异步索引表依然不断写入。
(待探索)
3、phoenix同步索引超时问题:Error: Operation timed out. (state=TIM01,code=6000)
需要在hbase/conf及phoenix/bin下的hbase-site.xml同时配置如下配置:
<!-- 指定查询在客户端上超时的毫秒数。 默认值是10分钟。-->
<property>
<name>phoenix.query.timeoutMs</name>
<value>1200000</value>
</property>
<!-- 当线程数大于客户端线程池数量,线程在终止之前等待新任务的最长时间(以毫秒为单位)。默认值为60秒 -->
<property>
<name>phoenix.query.keepAliveMs</name>
<value>1200000</value>
</property>
<!-- 默认60s,该参数表示一次RPC请求的超时时间。RPC时间超过该值,客户端就会主动关闭socket -->
<property>
<name>hbase.rpc.timeout</name>
<value>1200000</value>
</property>
<!-- regionserer租约时间,默认值是60s -->
<property>
<name>hbase.regionserver.lease.period</name>
<value>1200000</value>
</property>
<!-- 默认20min,该参数表示HBase客户端发起一次数据操作直至得到响应之间总的超时时间 -->
<property>
<name>hbase.client.operation.timeout</name>
<value>1200000</value>
</property>
<!-- scan缓存,默认为2147483647,不可太大,避免占用过多内存。 -->
<property>
<name>hbase.client.scanner.caching</name>
<value>1000</value>
</property>
<!-- 默认60s,该参数是表示HBase客户端发起一次scan操作的rpc调用至得到响应之间总的超时时间 -->
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>1200000</value>
</property>
4、phoenix构建索引时间戳类型的索引列发生异常 ERROR 201 (22000): Illegal data
Error: ERROR 201 (22000): Illegal data. (state=22000,code=201)
java.sql.SQLException: ERROR 201 (22000): Illegal data.
at org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:493)
at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
at org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:189)
at org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:173)
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:140)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1298)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1239)
at org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(RoundRobinResultIterator.java:176)
at org.apache.phoenix.iterate.RoundRobinResultIterator.next(RoundRobinResultIterator.java:91)
at org.apache.phoenix.compile.UpsertCompiler$ClientUpsertSelectMutationPlan.execute(UpsertCompiler.java:1318)
at org.apache.phoenix.compile.DelegateMutationPlan.execute(DelegateMutationPlan.java:31)
at org.apache.phoenix.compile.PostIndexDDLCompiler$1.execute(PostIndexDDLCompiler.java:117)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.updateData(ConnectionQueryServicesImpl.java:3663)
at org.apache.phoenix.schema.MetaDataClient.buildIndex(MetaDataClient.java:1382)
at org.apache.phoenix.schema.MetaDataClient.alterIndex(MetaDataClient.java:3998)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAlterIndexStatement$1.execute(PhoenixStatement.java:1230)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:406)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:389)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:388)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:376)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1789)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:813)
at sqlline.SqlLine.begin(SqlLine.java:686)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:291)