前提：配置好机器名以、JDK环境及安装包

所有配置文件路径：$HADOOP_HOME/etc/hadoop

配置环境变量

  1. 打开/etc/profile文件

  2. 添加以下内容：vi /etc/profile

    1. export HADOOP_HOME=/usr/local/hadoop-3.2.1
    2. export PATH=$HADOOP_HOME/sbin:$HADOOP_HOME/bin:$PATH

  3. 测试环境变量：hadoop version

    [root@hadoop101 ~]# hadoop version
Hadoop 3.2.1
Source code repository https://gitbox.apache.org/repos/asf/hadoop.git -r b3cbbb467e22ea829b3808f4b7b01d07e0bf3842
Compiled by rohithsharmaks on 2019-09-10T15:56Z
Compiled with protoc 2.5.0
From source with checksum 776eaf9eee9c0ffc370bcbc1888737
This command was run using /usr/local/hadoop-3.2.1/share/hadoop/common/hadoop-common-3.2.1.jar

配置core-site.xml

  1. 进入配置文件目录：$HADOOP_HOME/etc/hadoop

    cd $HADOOP_HOME/etc/hadoop

  2. 编辑core-site.xml文件：vi core-site.xml

  3. 添加以下配置信息：
    <configuration>
 <property>
     <name>fs.defaultFS</name>
     <value>hdfs://hadoop101:9000</value>
 </property>
 <property>
     <name>hadoop.tmp.dir</name>
     <value>/home/data/hadoop/tmp</value>
 </property>
</configuration>
    注意：
  • defaultFS中的值hadoop101为当前主机的主机名
  • tmp.dir为数据临时目录，需保证父目录存在

配置hdfs-site.xml

  1. 进入配置文件目录：$HADOOP_HOME/etc/hadoop

    cd $HADOOP_HOME/etc/hadoop

  2. 编辑hdfs-site.xml文件：vi hdfs-site.xml

  3. 添加以下配置信息：
    <configuration>
 <property>
     <name>dfs.replication</name>
     <value>1</value>
 </property>
</configuration>
    注意：
  • dfs.replication：表示复本的数量

配置yarn-site.xml

  1. 进入配置文件目录：$HADOOP_HOME/etc/hadoop

    cd $HADOOP_HOME/etc/hadoop

  2. 编辑hdfs-site.xml文件：vi yarn-site.xml

  3. 添加以下配置信息：
    <configuration>
 <property>
     <name>yarn.nodemanager.aux-services</name>
     <value>mapreduce_shuffle</value>
 </property>
 <property>
     <name>yarn.nodemanager.env-whitelist</name>
     <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
 </property>
 <property>
     <name>yarn.resourcemanager.hostname</name>
     <value>hadoop101</value>
 </property>
</configuration>
    注意：
  • yarn.nodemanager.aux-services: Shuffle service that needs to be set for Map Reduce applications.
  • yarn.nodemanager.env-whitelist: Environment properties to be inherited by containers from NodeManagers

配置hadoop-env.sh

  1. 进入配置文件目录：$HADOOP_HOME/etc/hadoop

    cd $HADOOP_HOME/etc/hadoop

  2. 编辑hdfs-site.xml文件：vi hadoop-env.sh

  3. 添加以下配置信息：
    export JAVA_HOME=/usr/local/jdk1.8.0_144
export HADOOP_LOG_DIR=/home/data/hadoop/logs