HbaseonYarn-编译部署slider
1、编译Slider,
成都创新互联总部坐落于成都市区,致力网站建设服务有网站建设、成都做网站、网络营销策划、网页设计、网站维护、公众号搭建、小程序开发、软件开发等为企业提供一整套的信息化建设解决方案。创造真正意义上的网站建设,为互联网品牌在互动行销领域创造价值而不懈努力!
[root@plat1 apache-slider-0.91.0-incubating]# pwd /usr/local/src/apache-slider-0.91.0-incubating
因采用CDH版本hadoop(2.6.0-cdh6.9.0),hbase(1.2.0-cdh6.9.0),修改该目录下pom文件
2.6.0-cdh6.9.0 1.2.0-cdh6.9.0 1.7.0
注释掉slider-core和slider-funtest中对hadoop-minicluster包依赖
跳过maven测试,执行编译命令
[root@plat1 apache-slider-0.91.0-incubating]# mvn clean package -Dmaven.test.skip=true -DskipTests
[INFO] Slider ............................................. SUCCESS [ 0.762 s]
[INFO] Command Logger Sample Application .................. SUCCESS [ 0.971 s]
[INFO] Slider Command Logger Slider App Package ........... SUCCESS [ 0.766 s]
[INFO] Slider Core ........................................ SUCCESS [ 10.766 s]
[INFO] Slider Agent ....................................... SUCCESS [ 1.245 s]
[INFO] Apache Slider Binary Distribution .................. SUCCESS [ 21.933 s]
[INFO] Slider Functional Tests ............................ SUCCESS [ 5.285 s]
[INFO] Slider App Packages ................................ SUCCESS [ 0.012 s]
[INFO] Slider Accumulo App Package ........................ SUCCESS [ 5.578 s]
[INFO] Slider HBase App Package ........................... SUCCESS [ 0.017 s]
[INFO] Slider Kafka App Package ........................... SUCCESS [ 1.046 s]
[INFO] Slider Storm App Package ........................... SUCCESS [ 0.011 s]
[INFO] Slider Tomcat App Package Testing WAR file ......... SUCCESS [ 0.886 s]
[INFO] Slider Tomcat App Package .......................... SUCCESS [ 1.666 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51.578 s
[INFO] Finished at: 2016-12-01T12:00:14+08:00
[INFO] Final Memory: 89M/2916M
[INFO] ------------------------------------------------------------------------
2、下载hbase-1.2.0-cdh6.9.0.tar.gz,存放目录/usr/local/src
wget http://archive.cloudera.com/cdh6/cdh/5/hbase-1.2.0-cdh6.9.0.tar.gz
3、生成slider-hbase-app-package-1.2.0-cdh6.9.0.zip
mvn clean package -P hbase-app-package -D pkg.version=1.2.0-cdh6.9.0 -Dpkg.name=hbase-1.2.0-cdh6.9.0.tar.gz -D pkg.src=/usr/local/src
-Dhbase.version=
-Dpkg.version=
-Dpkg.name=
-Dpkg.src=
4、将slider编译成功的包部署,配置环境变量
成功编译的包位于SLIDER_SRC_HOME/slider-assembly/target/slider-0.91.0-incubating-all.tar.gz
解压至/usr/local/slider
并配置环境变量export SLIDER_HOME=/usr/local/slider
5、创建package
slider package --install --name HBASE --package slider-0.91.0-incubating/slider-hbase-app-package-1.2.0-cdh6.9.0.zip
验证:
[hadoop@nn1 slider]$ hadoop fs -ls /user/hadoop/.slider/package/HBASE/slider-hbase-app-package-1.2.0-cdh6.9.0.zip -rw-r--r-- 3 hadoop hadoop 241347310 2016-12-01 13:43 /user/hadoop/.slider/package/HBASE/slider-hbase-app-package-1.2.0-cdh6.9.0.zip
6、提取slider-hbase-app-package-1.2.0-cdh6.9.0.zip中appConfig-default.json和resource-default.json
unzip slider-hbase-app-package-1.2.0-cdh6.9.0.zip appConfig-default.json -d /usr/local/slieder/
unzip slider-hbase-app-package-1.2.0-cdh6.9.0.zip resource-default.json -d /usr/local/slider/
7、修改appConfig-default.json
{ "schema": "http://example.org/specification/v2.0.0", "metadata": { }, "global": { "application.def": ".slider/package/HBASE/slider-hbase-app-package-1.2.0-cdh6.9.0.zip", "create.default.zookeeper.node": "true", "java_home": "/usr/local/jdk", "system_configs": "core-site", "site.global.app_user": "hadoop", "site.global.app_version": "1.2.0-cdh6.9.0", "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/hbase-1.2.0-cdh6.9.0", "site.global.metric_collector_host": "${NN_HOST}", "site.global.metric_collector_port": "6188", "site.global.metric_collector_lib": "", "site.global.hbase_instance_name": "instancename", "site.global.hbase_root_password": "secret", "site.global.user_group": "hadoop", "site.global.monitor_protocol": "http", "site.global.hbase_thrift_port": "${HBASE_THRIFT.ALLOCATED_PORT}", "site.global.hbase_thrift2_port": "${HBASE_THRIFT2.ALLOCATED_PORT}", "site.global.hbase_rest_port": "${HBASE_REST.ALLOCATED_PORT}", "site.hbase-env.hbase_master_heapsize": "1024m", "site.hbase-env.hbase_regionserver_heapsize": "1024m", "site.hbase-site.hbase.rootdir": "${DEFAULT_DATA_DIR}/data", "site.hbase-site.hbase.superuser": "${USER_NAME}", "site.hbase-site.hbase.tmp.dir": "${AGENT_WORK_ROOT}/work/app/tmp", "site.hbase-site.hbase.local.dir": "${hbase.tmp.dir}/local", "site.hbase-site.hbase.zookeeper.quorum": "${ZK_HOST}", "site.hbase-site.zookeeper.znode.parent": "${DEFAULT_ZK_PATH}", "site.hbase-site.hbase.regionserver.info.port": "0", "site.hbase-site.hbase.bulkload.staging.dir": "/user/${USER_NAME}/hbase-staging", "site.hbase-site.hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint", "site.hbase-site.hbase.master.info.port": "${HBASE_MASTER.ALLOCATED_PORT}", "site.hbase-site.hbase.regionserver.port": "0", "site.hbase-site.hbase.master.port": "0" }, "components": { "slider-appmaster": { "jvm.heapsize": "1024M" } } }
8、修改resource-default.json
{ "schema": "http://example.org/specification/v2.0.0", "metadata": { }, "global": { "yarn.log.include.patterns": "", "yarn.log.exclude.patterns": "", "yarn.component.instances": "1", "yarn.vcores": "1" }, "components": { "slider-appmaster": { "yarn.memory": "10240" }, "HBASE_MASTER": { "yarn.role.priority": "1", "yarn.component.instances": "1", "yarn.placement.escalate.seconds": "10", "yarn.memory": "15000" }, "HBASE_REGIONSERVER": { "yarn.role.priority": "2", "yarn.component.instances": "1", "yarn.memory": "15000", "yarn.container.failure.threshold": "15", "yarn.placement.escalate.seconds": "60" }, "HBASE_REST": { "yarn.role.priority": "3", "yarn.component.instances": "1", "yarn.component.placement.policy": "1", "yarn.memory": "5560" }, "HBASE_THRIFT": { "yarn.role.priority": "4", "yarn.component.instances": "1", "yarn.component.placement.policy": "1", "yarn.memory": "5560" }, "HBASE_THRIFT2": { "yarn.role.priority": "5", "yarn.component.instances": "1", "yarn.component.placement.policy": "1", "yarn.memory": "5560" } } }
9、启动hbase应用on yarn
[hadoop@nn1 slider]$ slider create hbase1 --template ./appConfig-default.json --resources ./resources-default.json
文章题目:HbaseonYarn-编译部署slider
网页URL:http://myzitong.com/article/ighgii.html