hbase与mapduce集成

1.从HBase读取数据
2.将数据写入HBase
3.结合以上两种,用于数据迁移

关联jar包
问题:hadoop环境变量中缺少hbase的jar包
方法一:
全部copy进去
可能会引起jar包之间的冲突
方法二:
将需要的hbase的jar包放入Hadoop的运行环境变量中
export HBASE_HOME=/opt/moduels/hbase-0.98.6-hadoop2
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_HOME/bin/hbase mapredcp
检验
/opt/moduels/hadoop-2.5.0/bin/yarn jar lib/hbase-server-0.98.6-hadoop2.jar rowcounter ns1:t1

模拟场景

场景构建
表一:
create 'stu_info','info','addinfo','work'
put 'stu_info','20180219_14001','info:name','tom'
put 'stu_info','20180219_14001','info:age','18'
put 'stu_info','20180219_14001','info:sex','male'
put 'stu_info','20180219_14001','addinfo:height','180'
put 'stu_info','20180219_14001','work:job','worker'
put 'stu_info','20180219_14002','info:name','rose'
put 'stu_info','20180219_14002','info:age','22'
put 'stu_info','20180219_14002','info:sex','female'
put 'stu_info','20180219_14002','addinfo:height','175'
put 'stu_info','20180219_14003','info:name','lee'
put 'stu_info','20180219_14003','info:age','22'
put 'stu_info','20180219_14004','info:name','tony'
put 'stu_info','20180219_14004','info:age','18'
put 'stu_info','20180219_14005','info:name','jim'
put 'stu_info','20180219_14005','info:age','19'
put 'stu_info','20180219_14006','info:name','xiaoming'
put 'stu_info','20180219_14006','info:age','20'
put 'stu_info','20180219_14006','addinfo:height','200'
表二:
create 'stu_result','info'
环境变量
修改hadoop下hadoop——env.sh文件
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/moduels/hbase-0.98.6-hadoop2/lib/*

©著作权归作者所有,转载或内容合作请联系作者
【社区内容提示】社区部分内容疑似由AI辅助生成,浏览时请结合常识与多方信息审慎甄别。
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

相关阅读更多精彩内容

友情链接更多精彩内容