Hadoop配置

• core-site.xml

路径：/opt/bigdata/hadoop/etc/hadoop

• hadoop-env.sh

路径：/opt/bigdata/hadoop/etc/hadoop
//声明一下jdk位置
export JAVA_HOME=/opt/bigdata/jdk

• hdfs-site.xml

路径：/opt/bigdata/hadoop/etc/hadoop



dfs.namenode.name.dir
/opt/bigdata/hadoop/tmp/hdfs/name



dfs.blocksize
268435456


dfs.namenode.handler.count
100



dfs.namenode.data.dir
/opt/bigdata/hadoop/tmp/hdfs/data



dfs.replicationr
3

• mapred-site.xml

路径：/opt/bigdata/hadoop/etc/hadoop



mapreduce.framework.name
yarn


yarn.app.mapreduce.am.env
HADOOP_MAPRED_HOME=/opt/bigdata/hadoop


mapreduce.map.env
HADOOP_MAPRED_HOME=/opt/bigdata/hadoop


mapreduce.reduce.env
HADOOP_MAPRED_HOME=/opt/bigdata/hadoop

• yarn-site.xml

路径：/opt/bigdata/hadoop/etc/hadoop



yarn.resourcemanager.hostname
bd01



yarn.nodemanager.aux-services
mapreduce_shuffle

• datanode（也有可能叫workers，根据版本查看）

路径：/opt/bigdata/hadoop/etc/hadoop
内容：bd01 bd02 bd03

• start-dfs.sh，stop-dfs.sh

路径：/opt/bigdata/hadoop/sbin
HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root

• start-yarn.sh ，stop-yarn.sh

路径：/opt/bigdata/hadoop/sbin
YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=root
YARN_NODEMANAGER_USER=root

• 在namenode格式化文件系统

hdfs namenode -format

• 开启分布式文件系统

start-all.sh