kerberos - 如何在CDP集群启用Kerberos - 《大数据》

1.文档编写目的

本文档讲述如何在CDP集群启用及配置Kerberos，您将学习到以下知识：
1.如何安装及配置KDC服务
2.如何通过CDP启用Kerberos
3.如何登录Kerberos并访问Hadoop相关服务

文档主要分为以下几步：
1.安装及配置KDC服务
2.CDP集群启用Kerberos
3.Kerberos使用

这篇文档将重点介绍如何在CDH集群启用及配置Kerberos，并基于以下假设：
1.CDP集群运行正常
2.集群未启用Kerberos
3.mariadb

以下是本次测试环境，但不是本操作手册的必需环境：
1.操作系统：redhat7.6
2.CDP和CM版本为7.1.4
3.采用root用户进行操作

2.KDC服务安装及配置
本文档中将KDC服务安装在Cloudera Manager Server所在服务器上（KDC服务可根据自己需要安装在其他服务器）
1.在Cloudera Manager服务器上安装KDC服务

[root@dgnode009 ~]# yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation

2.修改/etc/krb5.conf配置

[root@dgnode009 ~]# vim /etc/krb5.conf
# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm=IDGVALUE.COM
 dns_lookup_realm = false
 dns_lookup_kdc=false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 pkinit_anchors = /etc/pki/tls/certs/ca-bundle.crt
# default_realm = EXAMPLE.COM
 default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 IDGVALUE.COM = {
  kdc = dgnode009.idgvalue.com
  admin_server = dgnode009.idgvalue.com
 }

[domain_realm]
 .idgvalue.com = IDGVALUE.COM
 idgvalue.com = IDGVALUE.COM
~

3.修改/var/kerberos/krb5kdc/kadm5.acl配置

[root@dgnode009 ~]# vim /var/kerberos/krb5kdc/kadm5.acl
[root@dgnode009 ~]# cat /var/kerberos/krb5kdc/kadm5.acl
*/admin@IDGVALUE.COM    *

4.修改/var/kerberos/krb5kdc/kdc.conf配置

[root@dgnode009 ~]# vim /var/kerberos/krb5kdc/kdc.conf
[root@dgnode009 ~]# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
  IDGVALUE.COM = {
  max_renewable_life= 7d 0h 0m 0s
  #master_key_type = aes256-cts
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

5.创建Kerberos数据库，此处需要输入Kerberos数据库的密码。

[root@dgnode009 ~]# kdb5_util create –r IDGVALUE.COM -s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'IDGVALUE.COM',
master key name 'K/M@IDGVALUE.COM'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:输入密码
Re-enter KDC database master key to verify:

6.创建Kerberos的管理账号，需要输入管理员密码

[root@dgnode009 ~]# kadmin.local
Authenticating as principal root/admin@IDGVALUE.COM with password.
kadmin.local:  addprinc admin/admin@IDGVALUE.COM
WARNING: no policy specified for admin/admin@IDGVALUE.COM; defaulting to no policy
Enter password for principal "admin/admin@IDGVALUE.COM":
Re-enter password for principal "admin/admin@IDGVALUE.COM":
Principal "admin/admin@IDGVALUE.COM" created.
kadmin.local:  exit

7.将Kerberos服务添加到自启动服务，并启动krb5kdc和kadmin服务

[root@dgnode009 ~]# chkconfig krb5kdc on
注意：正在将请求转发到“systemctl enable krb5kdc.service”。
Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service.
[root@dgnode009 ~]# chkconfig kadmin on
注意：正在将请求转发到“systemctl enable kadmin.service”。
Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service.
[root@dgnode009 ~]# systemctl restart krb5kdc
[root@dgnode009 ~]# service krb5kdc start
Redirecting to /bin/systemctl start krb5kdc.service
[root@dgnode009 ~]# service kadmin start
Redirecting to /bin/systemctl start kadmin.service

8.测试Kerberos的管理员账号

[root@dgnode009 ~]# kinit admin/admin@IDGVALUE.COM
Password for admin/admin@IDGVALUE.COM:
[root@dgnode009 ~]# klist
Ticket cache: KEYRING:persistent:0:0
Default principal: admin/admin@IDGVALUE.COM

Valid starting       Expires              Service principal
2021-08-09T02:41:40  2021-08-10T02:41:40  krbtgt/IDGVALUE.COM@IDGVALUE.COM
        renew until 2021-08-16T02:41:40

9.为集群安装所有Kerberos客户端，包括Cloudera Manager

[root@dgnode009 ~]# pssh -h /node.list -i 'yum -y install krb5-libs krb5-workstation'

10.在Cloudera Manager Server服务器上安装额外的包

[root@dgnode009 ~]# yum -y install openldap-clients

11.将KDC Server上的/etc/krb5.conf文件拷贝到所有Kerberos客户端

[root@dgnode009 ~]# pscp -h /node.list /etc/krb5.conf  /etc/
[1] 02:48:31 [SUCCESS] root@10.168.1.109:22
[2] 02:48:31 [SUCCESS] root@10.168.1.110:22
[3] 02:48:31 [SUCCESS] root@10.168.1.111:22
[4] 02:48:31 [SUCCESS] root@10.168.1.112:22
[5] 02:48:31 [SUCCESS] root@10.168.1.113:22

3.CDH集群启用Kerberos
1.在KDC中给Cloudera Manager添加管理员账号

[root@dgnode009 ~]# kadmin.local
Authenticating as principal admin/admin@IDGVALUE.COM with password.
kadmin.local:  addprinc cloudera-scm/admin@IDGVALUE.COM
WARNING: no policy specified for cloudera-scm/admin@IDGVALUE.COM; defaulting to no policy
Enter password for principal "cloudera-scm/admin@IDGVALUE.COM":
Re-enter password for principal "cloudera-scm/admin@IDGVALUE.COM":
Principal "cloudera-scm/admin@IDGVALUE.COM" created.
kadmin.local:  exit

2.进入Cloudera Manager的“管理”-> “安全”界面

3.选择“启用Kerberos”，进入如下界面
确保如下列出的所有检查项都已完成

4.点击“继续”，配置相关的KDC信息，包括类型、KDC服务器、KDC Realm、加密类型以及待创建的Service Principal（hdfs，yarn,，hbase，hive等）的更新生命期等

5.不建议让Cloudera Manager来管理krb5.conf, 点击“继续”

6.输入Cloudera Manager的Kerbers管理员账号，必须和之前创建的账号一致，点击“继续”

7.等待启用Kerberos完成，点击“继续”

8.点击“继续”

9.等待集群重启成功，点击“继续”，至此已成功启用Kerberos。

4.Kerberos使用
使用dejin用户运行MapReduce任务及操作Hive，需要在集群所有节点创建dejin用户。
1.所有节点创建dejin用户

[root@dgnode009 ~]# pssh -h /node.list -i 'useradd dejin'

2.使用kadmin创建一个dejin的principal


[root@dgnode009 ~]# kadmin.local
Authenticating as principal admin/admin@IDGVALUE.COM with password.
kadmin.local:  addprinc dejin@IDGVALUE.COM
WARNING: no policy specified for dejin@IDGVALUE.COM; defaulting to no policy
Enter password for principal "dejin@IDGVALUE.COM":
Re-enter password for principal "dejin@IDGVALUE.COM":
Principal "dejin@IDGVALUE.COM" created.

3.c使用dejin用户登录Kerberos

[root@dgnode009 ~]# kdestroy
[root@dgnode009 ~]# kinit dejin
Password for dejin@IDGVALUE.COM:
[root@dgnode009 ~]# klist
Ticket cache: KEYRING:persistent:0:0
Default principal: dejin@IDGVALUE.COM

Valid starting       Expires              Service principal
2021-08-09T03:18:28  2021-08-10T03:18:28  krbtgt/IDGVALUE.COM@IDGVALUE.COM
        renew until 2021-08-16T03:18:28

3.运行MapReduce作业

[root@dgnode009 lib]# hadoop jar /opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p0.6300266/jars/hadoop-examples.jar pi 10 1
Number of Maps  = 10
Samples per Map = 1
Wrote input for Map #0
......
Wrote input for Map #9
Starting Job
......
2021-08-09 03:22:32,839 INFO mapred.LocalJobRunner: Finishing task: attempt_local1551488908_0001_r_000000_0
2021-08-09 03:22:32,839 INFO mapred.LocalJobRunner: reduce task executor complete.
2021-08-09 03:22:33,644 INFO mapreduce.Job:  map 100% reduce 100%
2021-08-09 03:22:33,645 INFO mapreduce.Job: Job job_local1551488908_0001 completed successfully
.........
Job Finished in 4.22 seconds
Estimated value of Pi is 3.60000000000000000000