CentOS7_Sqoop-1.4.6

资源规划

组件 LTSR003
OS centos7.6
JDK jvm
MySQL N.A
HDFS DataNode/HTTPFS
YARN NodeManager
Hive HiveServer2/Metastore/CLI/Beeline

安装介质

版本:sqoop-1.4.6.binhadoop-2.0.4-alpha.tar.gz
下载:[http://archive.apache.org/dist/sqoop/1.4.6/sqoop-1.4.6.bin
hadoop-2.0.4-alpha.tar.gz](http://archive.apache.org/dist/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz)

环境准备

安装Hadoop

参考:《CentOS7.6-安装Hadoop-2.7.2

安装MySQL

参考:《CentOS7.6-安装MySQL-5.7.30

安装Sqoop

  1. 根据资源规划,Sqoop安装在节点**LTSR003**上。

解压缩

cd modules
wget http://archive.apache.org/dist/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
tar -zvxf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C ~/modules/
rm sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

配置

配置sqoop-env.sh:

cd ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/conf
cp sqoop-env-template.sh sqoop-env.sh
vi ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/conf/sqoop-env.sh

配置如下:

# 末尾添加
export HADOOP_COMMON_HOME=/home/bigdata/modules/hadoop-2.7.2
export HADOOP_MAPRED_HOME=/home/bigdata/modules/hadoop-2.7.2
export HIVE_HOME=/home/bigdata/modules/apache-hive-2.3.4-bin

加载JDBC驱动

拷贝jdbc驱动到sqoop的lib目录下,如:

cp mysql-connector-java-5.1.47.jar ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/lib/

验证

cd ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha
bin/sqoop help

测试Sqoop是否能够成功连接数据库。

cd ~/modules/sqoop-1.4.6.bin__hadoop-2.0.4-alpha
bin/sqoop list-databases --connect jdbc:mysql://10.8.0.101.:3306/ --username root --password 123456