三、集群安装

下载地址：https://mirror.bit.edu.cn/apache/hadoop/common/

一、网络环境准备

采用NAT方式联网

二、服务器设置

同步时间
设置主机名
配置内网映射

配置ssh免密登录
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.198.131
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.198.128
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.198.129

防火墙设置
关闭防火墙 - service iptables stop
关闭开机启动 - chkconfig iptables off
查看是否关闭开机启动 - chkconfig iptables —list

三、集群节点规划

hdfs：namenode、SecondaryNamenode datanode
yarn：ResourceManager NodeManager
mapreduce

四、配置文件修改

• 安装配置jdk1.8

• 进入etc/hadoop目录下，配置hadoop的jdk变量，vim hadoop-env.sh

   ![image.png](https://cdn.nlark.com/yuque/0/2020/png/1161651/1585019717726-55402021-2080-4143-8914-adb40a512a1f.png#align=left&display=inline&height=35&margin=%5Bobject%20Object%5D&name=image.png&originHeight=69&originWidth=629&size=14819&status=done&style=none&width=314.5)

• 修改core-site.xml

   ![image.png](https://cdn.nlark.com/yuque/0/2020/png/1161651/1585019835917-2d94c88b-d648-4d3f-895b-09a822ff240c.png#align=left&display=inline&height=173&margin=%5Bobject%20Object%5D&name=image.png&originHeight=345&originWidth=863&size=86817&status=done&style=none&width=431.5)

• 修改hdfs-site.xml

   ![image.png](https://cdn.nlark.com/yuque/0/2020/png/1161651/1585019920464-75392e69-0a71-48c6-b35c-a9a88e5260c3.png#align=left&display=inline&height=166&margin=%5Bobject%20Object%5D&name=image.png&originHeight=332&originWidth=823&size=70871&status=done&style=none&width=411.5)

• 修改mapred-site.xml

   ![image.png](https://cdn.nlark.com/yuque/0/2020/png/1161651/1585020067246-3930aade-25c9-4fe7-bca4-79da01b0530d.png#align=left&display=inline&height=90&margin=%5Bobject%20Object%5D&name=image.png&originHeight=179&originWidth=639&size=34478&status=done&style=none&width=319.5)

• 修改yarn-site.xml

   ![image.png](https://cdn.nlark.com/yuque/0/2020/png/1161651/1585020280110-88ff6fd9-17ea-45f6-aed6-b97ac5bd591e.png#align=left&display=inline&height=200&margin=%5Bobject%20Object%5D&name=image.png&originHeight=399&originWidth=779&size=83689&status=done&style=none&width=389.5)

• 配置hadoop环境变量

   ![image.png](https://cdn.nlark.com/yuque/0/2020/png/1161651/1585020436896-94cdc1cb-4a9a-4802-afab-de1b647c3921.png#align=left&display=inline&height=74&margin=%5Bobject%20Object%5D&name=image.png&originHeight=147&originWidth=1048&size=82257&status=done&style=none&width=524)

• 修改启动、停止脚本start-dfs.sh、stop-dfs.sh, 加在文件的最上面即可

• 修改启动、停止脚本start-yarn.sh、stop-yarn.sh，加在文件的最上面即可

  ![image.png](https://cdn.nlark.com/yuque/0/2020/png/1161651/1585020787166-f1191e11-ad18-4224-8775-f17f7dbd4964.png#align=left&display=inline&height=80&margin=%5Bobject%20Object%5D&name=image.png&originHeight=159&originWidth=580&size=43329&status=done&style=none&width=290)

• 将三台机械配置文件同步 scp命令拷贝至另外两台

  五、启动

  启动前格式化namenode - hdfs namenode -format
  在192.168.198.128节点上启动hdfs - shin/start-dfs.sh
  在192.168.198.129节点上启动yarm - sbin/start-yarn.sh

  六、启动结果

  
  
  

七、管理界面

查看namenode端口：netstat -antp | fgrep 12811
yarn管理界面：http://192.168.198.129:8088/cluster
hdfs管理界面：http://192.168.198.128:50070