ps: 路径自己定义,最好找一个不会被误删的路径
1、集群每个主机创建keystore file和certificate(在每个主机执行时修改命令中的ip,重复执行如下两个命令)
a.Create a keystore file:
mkdir /raoyi
/usr/jdk64/jdk1.8.0_112/bin/keytool -genkey
-alias “10.1.236.54”
-keyalg RSA
-dname “cn=10.1.236.54,ou=asiainfo,o=asiainfo,l=Beijing,st=Beijing,c=CN”
-keypass 123456
-keystore “/raoyi/thisnode.keystore”
-storepass 123456
-validity 3650
b.Create a certificate:
/usr/jdk64/jdk1.8.0_112/bin/keytool -export
-alias “10.1.236.54”
-keystore “/raoyi/thisnode.keystore”
-rfc -file “/raoyi/10.1.236.54.crt”
-storepass 123456
2、文件赋权
chmod 755 /raoyi/*
3、拷贝上面生成的集群所有节点的.crt文件到Yarn ResourceManager主机的/raoyi目录下,重复执行如下命令(每次执行命令修改ip)生成一个包括所有主机的证书信息的all_truststore.keystore文件
/usr/jdk64/jdk1.8.0_112/bin/keytool -import
-noprompt
-alias “10.1.236.52”
-file “/raoyi/10.1.236.52.crt”
-keystore “/raoyi/all_truststore.keystore”
-storepass 123456
4、检查是否包含所有主机证书:
/usr/jdk64/jdk1.8.0_112/bin/keytool -list -keystore /raoyi/all_truststore.keystore -storepass 123456
5、把上面生成的/raoyi/all_truststore.keystore拷贝到集群所有节点的/raoyi目录下,并赋权chmod 755 /raoyi/all_truststore.keystore
6、如果需要使用spark2.x时,执行如下命令(spark1.x不需要执行,spark1.x不支持WebUI Encryption):
在所有Yarn ResourceManager主机上,添加所有节点的证书信息到java cacerts.
/usr/jdk64/jdk1.8.0_112/bin/keytool -import -noprompt -alias 10.1.236.57 -file “/raoyi/10.1.236.57.crt” -keystore /usr/jdk64/jdk1.8.0_112/jre/lib/security/cacerts -storepass changeit
/usr/jdk64/jdk1.8.0_112/bin/keytool -import -noprompt -alias 10.1.236.55 -file “/raoyi/10.1.236.55.crt” -keystore /usr/jdk64/jdk1.8.0_112/jre/lib/security/cacerts -storepass changeit
/usr/jdk64/jdk1.8.0_112/bin/keytool -import -noprompt -alias 10.1.236.56 -file “/raoyi/10.1.236.56.crt” -keystore /usr/jdk64/jdk1.8.0_112/jre/lib/security/cacerts -storepass changeit
7、检查导入java cacerts中的主机证书信息
/usr/jdk64/jdk1.8.0_112/bin/keytool -list -keystore /usr/jdk64/jdk1.8.0_112/jre/lib/security/cacerts -storepass changeit | grep -i -E “ocdp|236”
8、如果配置了external shuffle service(spark.shuffle.service.enabled=true),则在yarn-site.xml中添加如下属性:
spark.authenticate
true
9、通过ambari在spark-defaults和spark2-defaults中添加如下属性:
spark.acls.enable=true
spark.admin.acls=*
spark.authenticate=true
spark.authenticate.enableSaslEncryption=true
spark.ssl.enabled=true
spark.ssl.enabledAlgorithms=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA
spark.ssl.keyPassword=123456
spark.ssl.keyStore=/raoyi/thisnode.keystore
spark.ssl.keyStorePassword=123456
spark.ssl.protocol=TLS
spark.ssl.trustStore=/raoyi/all_truststore.keystore
spark.ssl.trustStorePassword=123456
spark.ssl.needClientAuth=true
spark.history.ui.acls.enable=true
10、重启相应受影响的服务