大家好，我是怀瑾握瑜，一只大数据萌新，家有两只吞金兽，嘉与嘉，上能code下能teach的全能奶爸 如果您喜欢我的文章，可以[关注⭐]+[点赞👍]+[评论📃]，您的三连是我前进的动力，期待与您共同成长~

1. IMPALA环境准备

1、上传IMPALA.tar.gz至/var/lib/ambari-server/resources/stacks/HDP/3.1/services并解压
2、上传IMPALA_CDH.tar.gz至/var/www/html并解压
3、上传impala.repo镜像至/etc/yum.repos.d/
网页段可以打开相应页面
http://172.29.30.61/IMPALA_CDH/
image.png

2. 重启ambari-server

  1. # service ambari-server restart

3. 安装ampala

1、登陆ambari页面管理
2、首页左下角点击add service，勾选未安装的impala进行安装
3、选择需要安装到的主机即可
image.png

安装可能会遇到一些报错，目前有人如下解决方案：
1）

  1. Traceback (most recent call last):
  2. File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 38, in <module>
  3. BeforeAnyHook().execute()
  4. File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
  5. method(env)
  6. File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 31, in hook
  7. setup_users()
  8. File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/shared_initialization.py", line 50, in setup_users
  9. groups = params.user_to_groups_dict[user],
  10. KeyError: u'impala'
  11. Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-863.json', '/
  12. ambari主节点执行命令行，需要有python运行环境，可以尝试使用默认的，报错就需要手动安装
  13. $cluster_name为ambari的cluster名称，这里为Sea
  14. $ambari_server为ambari地址，这里为172.29.30.61
  15. # cd /var/lib/ambari-server/resources/scripts
  16. # python configs.py -u admin -p admin -n $cluster_name -l $ambari_server -t 8080 -a get -c cluster-env |grep -i ignore_groupsusers_create
  17. "ignore_groupsusers_create": "false",
  18. # python configs.py -u admin -p admin -n $cluster_name -l $ambari_server -t 8080 -a set -c cluster-env -k ignore_groupsusers_create -v true

2）所有安装impala节点修改python文件

  1. # vim /var/lib/ambari-agent/cache/stacks/HDP/3.1/services/IMPALA/package/scripts/impala-catalog.py

把涉及地址的那三行代码删除，最终结果如下：
image.png
执行后重新安装

4. 修改HDFS相关配置文件

在ambari上操作，找到hdfs的config
找到Custom core-site，点击Add property
添加如下内容

  1. dfs.client.read.shortcircuit=true
  2. dfs.client.read.shortcircuit.skip.checksum=false
  3. dfs.datanode.hdfs-blocks-metadata.enabled=true

找到Custom hdfs-site,点击Add property

  1. dfs.datanode.hdfs-blocks-metadata.enabled=true
  2. dfs.block.local-path-access.user=impala如果报错的话就改成hadoop
  3. dfs.client.file-block-storage-locations.timeout.millis=60000

重启hdfs

5. 修改impala配置（所有机器）

  1. # vim /etc/default/impala

在MEM_LIMIT=20gb配置下添加kudu地址（前提为kudu已安装完成，但该步骤不影响impala整体安装）： 

  1. KUDU_MASTER_HOST=xxx3.hadoop.com:7051

在IMPALA_SERVER_ARGS 配置项最后添加如下配置：

  1. -kudu_master_hosts=${KUDU_MASTER_HOST} \

image.png

6. 启动impala前准备工作（所有节点）

1、查看/etc/impala/conf/下是否有core-site.xml hdfs-site.xml hive-site.xml
如果没有需要从hive和hdfs的conf中同步以上三个xml文件
2、创建impala目录并添加相关授权
各节点将impala用户添加到组中

  1. # usermod -G hive,hdfs,hadoop impala
  1. 创建impala相关目录并授权
  1. # su - hdfs -c "hadoop fs -mkdir /user/impala"
  2. # su - hdfs -c "hadoop fs -chown -R impala /user/impala"

3、配置bigtop-utils的JAVA_HOME

  1. # vim /etc/default/bigtop-utils
  2. export JAVA_HOME=/app/tools/java/jdk1.8.0_201

4、把hadoop低版本的jar包上传并创建软连接到/usr/lib/impala/lib
上传hdp-2.6.4.0-91_hadoop_lib.zip文件到/usr/hdp/并解压缩
执行如下软连接命令

  1. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-annotations.jar /usr/lib/impala/lib/hadoop-annotations.jar
  2. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-archives-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-archives.jar
  3. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-auth.jar /usr/lib/impala/lib/hadoop-auth.jar
  4. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-aws.jar /usr/lib/impala/lib/hadoop-aws.jar
  5. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-common.jar /usr/lib/impala/lib/hadoop-common.jar
  6. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-hdfs-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-hdfs.jar
  7. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-mapreduce-client-common-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-mapreduce-client-common.jar
  8. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-mapreduce-client-core-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-mapreduce-client-core.jar
  9. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-mapreduce-client-jobclient-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-mapreduce-client-jobclient.jar
  10. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-mapreduce-client-shuffle-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-mapreduce-client-shuffle.jar
  11. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-api-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-api.jar
  12. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-client-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-client.jar
  13. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-common-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-common.jar
  14. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-server-applicationhistoryservice-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-server-applicationhistoryservice.jar
  15. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-server-common-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-server-common.jar
  16. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-server-nodemanager-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-server-nodemanager.jar
  17. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-server-resourcemanager-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-server-resourcemanager.jar
  18. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop-yarn-server-web-proxy-2.7.3.2.6.4.0-91.jar /usr/lib/impala/lib/hadoop-yarn-server-web-proxy.jar
  19. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hbase-annotations-1.2.0-cdh5.16.2.jar /usr/lib/impala/lib/hbase-annotations.jar
  20. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hbase-client-1.2.0-cdh5.16.2.jar /usr/lib/impala/lib/hbase-client.jar
  21. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hbase-common-1.2.0-cdh5.16.2.jar /usr/lib/impala/lib/hbase-common.jar
  22. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hbase-protocol-1.2.0-cdh5.16.2.jar /usr/lib/impala/lib/hbase-protocol.jar
  23. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-ant-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-ant.jar
  24. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-beeline-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-beeline.jar
  25. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-common-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-common.jar
  26. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-exec-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-exec.jar
  27. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-hbase-handler-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-hbase-handler.jar
  28. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-metastore-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-metastore.jar
  29. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-serde-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-serde.jar
  30. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-service-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-service.jar
  31. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-shims-common-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-shims-common.jar
  32. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-shims-0.20S-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-shims.jar
  33. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hive-shims-scheduler-1.2.1000.2.6.4.0-91.jar /usr/lib/impala/lib/hive-shims-scheduler.jar
  34. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop/libhadoop.so /usr/lib/impala/lib/libhadoop.so
  35. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop/libhadoop.so.1.0.0 /usr/lib/impala/lib/libhadoop.so.1.0.0
  36. ln -s -f /usr/hdp/hdp-2.6.4.0-91_hadoop_lib/hadoop/libhdfs.so /usr/lib/impala/lib/libhdfs.so
  37. ln -s -f /usr/hdp/2.6.4.0-91/usr/lib/libhdfs.so.0.0.0 /usr/lib/impala/lib/libhdfs.so.0.0.0

7. impala启动及报错解决方案

通过ambari重启impala

结束语

如果您喜欢我的文章，可以[关注⭐]+[点赞👍]+[评论📃]，您的三连是我前进的动力，期待与您共同成长~ 可关注企鹅号【怀瑾握瑜的嘉与嘉】，获取资源下载方式