CentOS7_Sqoop-1.4.6
资源规划
组件 | LTSR003 |
---|---|
OS | centos7.6 |
JDK | jvm |
MySQL | N.A |
HDFS | DataNode/HTTPFS |
YARN | NodeManager |
Hive | HiveServer2/Metastore/CLI/Beeline |
安装介质
版本:sqoop-1.4.6.binhadoop-2.0.4-alpha.tar.gz
下载:[http://archive.apache.org/dist/sqoop/1.4.6/sqoop-1.4.6.binhadoop-2.0.4-alpha.tar.gz](http://archive.apache.org/dist/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz)
环境准备
安装Hadoop
安装MySQL
安装Sqoop
根据资源规划,Sqoop安装在节点**LTSR003**上。
解压缩
cd modules
wget http://archive.apache.org/dist/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
tar -zvxf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C ~/modules/
rm sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
配置
配置sqoop-env.sh:
cd ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/conf
cp sqoop-env-template.sh sqoop-env.sh
vi ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/conf/sqoop-env.sh
配置如下:
# 末尾添加
export HADOOP_COMMON_HOME=/home/bigdata/modules/hadoop-2.7.2
export HADOOP_MAPRED_HOME=/home/bigdata/modules/hadoop-2.7.2
export HIVE_HOME=/home/bigdata/modules/apache-hive-2.3.4-bin
加载JDBC驱动
拷贝jdbc驱动到sqoop的lib目录下,如:
cp mysql-connector-java-5.1.47.jar ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/lib/
验证
cd ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha
bin/sqoop help
测试Sqoop是否能够成功连接数据库。
cd ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha
bin/sqoop list-databases --connect jdbc:mysql://10.8.0.101.:3306/ --username root --password 123456