Ranger同步ldap组问题

时间:2022-07-27
本文章向大家介绍Ranger同步ldap组问题，主要内容包括其使用实例、应用技巧、基本知识点总结和需要注意事项，具有一定的参考价值，需要的朋友可以参考一下。

问题描述

按照我们之前的配置，在CDH7.1.1上为Ranger集成OpenLDAP认证这边文章中，我们为Ranger集成了OpenLDAP认证，刚开始给Hive、HDFS、HBase授权的时候，没发现有什么毛病，ldap用户都能正常登录和同步。后来使用Ranger API给用户批量授权时，将大量用户放在同一用户组里，比较好管理。这时我们才发现ldap的用户组没有被Ranger同步过来。

查看openldap，我们可以看到openldap已经创建etl_user用户和用户组。

# etl_user, Group, macro.com
dn: cn=etl_user,ou=Group,dc=macro,dc=com
objectClass: posixGroup
objectClass: top
cn: etl_user
userPassword:: e2NyeXB0fXg=
gidNumber: 50001
memberUid: etl_user

# etl_user, People, macro.com
dn: uid=etl_user,ou=People,dc=macro,dc=com
uid: etl_user
cn: etl_user
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
userPassword:: MTIzNDU2
shadowLastChange: 18235
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 50001
gidNumber: 50001
homeDirectory: /home/etl_user

# search result
search: 2
result: 0 Success

# numResponses: 143
# numEntries: 142

通过之前的配置，我们在ranger中也可以看到etl_user用户。

但是我们发现etl_user用户组却没有被同步过来。

问题分析

仔细查看下cloudera官网，发现ranger的ldap用户组配置有误，需要补充如下

修改配置，重启Ranger，发现还是无法同步ldap用户组。这时我们来看一看日志有没有相应的报错

我们可以看到类似的信息：

2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for etl_user
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for oozie
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for hbase
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for sentry
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for impala
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000

用ldap命令搜索etl_user的用户和用户组，看一下有什么问题：

etl_user用户

[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=People,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <ou=People,dc=macro,dc=com> with scope subtree
# filter: cn=etl_user
# requesting: ALL
#

# etl_user, People, macro.com
dn: uid=etl_user,ou=People,dc=macro,dc=com
uid: etl_user
cn: etl_user
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
userPassword:: MTIzNDU2
shadowLastChange: 18235
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 50001
gidNumber: 50001
homeDirectory: /home/etl_user

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

etl_user用户组

[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=Group,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <ou=Group,dc=macro,dc=com> with scope subtree
# filter: cn=etl_user
# requesting: ALL
#

# etl_user, Group, macro.com
dn: cn=etl_user,ou=Group,dc=macro,dc=com
objectClass: posixGroup
objectClass: top
cn: etl_user
userPassword:: e2NyeXB0fXg=
gidNumber: 50001

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

看样子etluser用户组似乎没有拥有etl_user这个用户。我们在etl_user用户组中添加memberUid这个属性

[root@cdh1 ~]# cat etl_user.ldif 
dn: cn=etl_user,ou=Group,dc=macro,dc=com
changetype: modify
add: memberUid
memberUid: etl_user

然后修改用户

[root@cdh1 ~]# ldapmodify -Z -x -W -D "cn=Manager,dc=macro,dc=com" -f etl_user.ldif 
Enter LDAP Password: 
modifying entry "cn=etl_user,ou=Group,dc=macro,dc=com"

再看看etl_user用户组，用户组属性已经被添加进去了。

[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=Group,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <ou=Group,dc=macro,dc=com> with scope subtree
# filter: cn=etl_user
# requesting: ALL
#

# etl_user, Group, macro.com
dn: cn=etl_user,ou=Group,dc=macro,dc=com
objectClass: posixGroup
objectClass: top
cn: etl_user
userPassword:: e2NyeXB0fXg=
gidNumber: 50001
memberUid: etl_user

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

再重启ranger,我们可以发现etl_user用户组已经被同步了

同时，日志的信息已经正确了

2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: longName: etl_user, userName: etl_user
2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No. of members in the group etl_user = 1
2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: LdapDeltaUserGroupBuilder.getGroups() completed with group count: 77
2020-09-22 00:38:18,281 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Unix Auth Service!
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [SSLv2Hello]
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1]
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1.1]
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1.2]
2020-09-22 00:38:28,755 INFO org.apache.ranger.ldapusersync.process.LdapPolicyMgrUserGroupBuilder: valid cookie saved 
2020-09-22 00:38:29,254 INFO org.apache.ranger.usergroupsync.UserGroupSync: End: initial load of user/group from source==>sink
2020-09-22 00:38:29,254 INFO org.apache.ranger.usergroupsync.UserGroupSync: Done initializing user/group source and sink

小问题

还有个小问题忘了提及，第一次启动Ranger时会出现以下错误，

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes ErrorCode: 1071
2020-09-17 11:04:39,363  [E] ranger_core_db_mysql.sql file import failed!
2020-09-17 11:04:39,364  [I] Unable to create DB schema, Please drop the database and try again

这是因为建立索引的时候，字符长度超过255，由于uef-8字符最长只能为255，超长之后会报错，所以需要对MySQL做如下配置:

set global innodb_file_per_table = on,
innodb_file_format = Barracuda,
innodb_large_prefix = on;

随机文章
本站知识点必读
最近更新