大數據安全利器 Ranger 編譯安裝
作者：數據庫干貨鋪
Ranger大數據領域的一個集中式安全管理框架，它可以對諸如Hdfs、Hive、Kafka、Storm等組件進行細粒度的權限控制。本文將介紹部署過程。
ranger大數據領域的一個集中式安全管理框架，它可以對諸如hdfs、hive、kafka、storm等組件進行細粒度的權限控制。本文將介紹部署過程
1、部署準備
ranger: 進入apach官網下載
http://ranger.apache.org/download.html， 本次使用的是ranger1.2.0 ，地址為http://mirror.bit.edu.cn/apache/ranger/1.2.0/apache-ranger-1.2.0.tar.gz。
maven: 進入Apache的maven官網
http://maven.apache.org/download.cgi下載, 本次用的是maven3.6 ，下載地址 http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz。
python2.7: 因編譯及試用中需要Python2.7版本的Python，因此如果為Centos6系統，需要手動升級Python至Python2.7，升級過程可參考歷史文章Python升級。
MySQL： 需要mysql數據庫，如無可用MySQL需要部署一套MySQL，部署方法請參考歷史文章MySQL部署
mysql-connector-java: 進入MySQL官網下載
???https://dev.mysql.com/downloads/connector/j/5.1.html。??
bc命令: 使用過程中需要使用bc命令：
## 下載ranger
wget http://mirror.bit.edu.cn/apache/ranger/1.2.0/apache-ranger-1.2.0.tar.gz
## 下載maven
wget http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz
## 安裝bc命令
yum install bc -y
2、編譯安裝過程
（1） 解壓
# 解壓ranger及maven包
tar -zxvf apache-ranger-1.2.0.tar.gz
tar -zxvf apache-maven-3.6.1-bin.tar.gz
# 將maven相關命令路徑配置到環境變量
vim /etc/profile
export PATH=/usr/local/maven/bin:$PATH
（2）編譯
cd apache-ranger-1.2.0
mvn clean
# 編譯
mvn clean compile package assembly:assembly install -DskipTests -Drat.skip=true編譯過程中會有如下信息
執行過程中會出現很多信息，且編譯時間很長，如果出現異常，根據對應的報錯信息進行處理，例如Python包不存在時需要手動添加；如果某個jar不存在，可以手動下載放置對應的路徑或修改pom地址或文件。
經過很長一段時間，且多次解決錯誤后，編譯完成，出現如下結果：
[INFO] Reactor Summary for ranger 1.2.0:
[INFO]
[INFO] ranger ............................................. SUCCESS [ 0.244 s]
[INFO] Jdbc SQL Connector ................................. SUCCESS [ 0.573 s]
[INFO] Credential Support ................................. SUCCESS [ 0.391 s]
[INFO] Audit Component .................................... SUCCESS [ 1.065 s]
[INFO] Common library for Plugins ......................... SUCCESS [ 1.332 s]
[INFO] Installer Support Component ........................ SUCCESS [ 0.153 s]
[INFO] Credential Builder ................................. SUCCESS [ 0.293 s]
[INFO] Embedded Web Server Invoker ........................ SUCCESS [ 0.345 s]
[INFO] Key Management Service ............................. SUCCESS [ 0.984 s]
[INFO] ranger-plugin-classloader .......................... SUCCESS [ 0.209 s]
[INFO] HBase Security Plugin Shim ......................... SUCCESS [ 0.830 s]
[INFO] HBase Security Plugin .............................. SUCCESS [ 1.092 s]
[INFO] Hdfs Security Plugin ............................... SUCCESS [ 0.786 s]
[INFO] Hive Security Plugin ............................... SUCCESS [ 1.135 s]
[INFO] Knox Security Plugin Shim .......................... SUCCESS [ 0.524 s]
[INFO] Knox Security Plugin ............................... SUCCESS [ 0.712 s]
[INFO] Storm Security Plugin .............................. SUCCESS [ 0.533 s]
[INFO] YARN Security Plugin ............................... SUCCESS [ 0.398 s]
[INFO] Ranger Util ........................................ SUCCESS [ 1.099 s]
[INFO] Unix Authentication Client ......................... SUCCESS [ 0.259 s]
[INFO] Security Admin Web Application ..................... SUCCESS [ 12.847 s]
[INFO] KAFKA Security Plugin .............................. SUCCESS [ 0.458 s]
[INFO] SOLR Security Plugin ............................... SUCCESS [ 1.208 s]
[INFO] NiFi Security Plugin ............................... SUCCESS [ 0.386 s]
[INFO] NiFi Registry Security Plugin ...................... SUCCESS [ 0.381 s]
[INFO] Unix User Group Synchronizer ....................... SUCCESS [ 1.016 s]
[INFO] Ldap Config Check Tool ............................. SUCCESS [ 0.222 s]
[INFO] Unix Authentication Service ........................ SUCCESS [ 0.330 s]
[INFO] KMS Security Plugin ................................ SUCCESS [ 0.490 s]
[INFO] Tag Synchronizer ................................... SUCCESS [ 1.211 s]
[INFO] Hdfs Security Plugin Shim .......................... SUCCESS [ 0.311 s]
[INFO] Hive Security Plugin Shim .......................... SUCCESS [ 0.800 s]
[INFO] YARN Security Plugin Shim .......................... SUCCESS [ 0.320 s]
[INFO] Storm Security Plugin shim ......................... SUCCESS [ 0.365 s]
[INFO] KAFKA Security Plugin Shim ......................... SUCCESS [ 0.313 s]
[INFO] SOLR Security Plugin Shim .......................... SUCCESS [ 0.701 s]
[INFO] Atlas Security Plugin Shim ......................... SUCCESS [ 0.398 s]
[INFO] KMS Security Plugin Shim ........................... SUCCESS [ 0.342 s]
[INFO] ranger-examples .................................... SUCCESS [ 0.042 s]
[INFO] Ranger Examples - Conditions and ContextEnrichers .. SUCCESS [ 0.335 s]
[INFO] Ranger Examples - SampleApp ........................ SUCCESS [ 0.140 s]
[INFO] Ranger Examples - Ranger Plugin for SampleApp ...... SUCCESS [ 0.322 s]
[INFO] Ranger Tools ....................................... SUCCESS [ 0.457 s]
[INFO] Atlas Security Plugin .............................. SUCCESS [ 0.722 s]
[INFO] Sqoop Security Plugin .............................. SUCCESS [ 0.472 s]
[INFO] Sqoop Security Plugin Shim ......................... SUCCESS [ 0.293 s]
[INFO] Kylin Security Plugin .............................. SUCCESS [ 0.410 s]
[INFO] Kylin Security Plugin Shim ......................... SUCCESS [ 0.315 s]
[INFO] Unix Native Authenticator .......................... SUCCESS [ 0.491 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:24 h
[INFO] Finished at: 2019-08-19T09:47:05+08:00
[INFO] ------------------------------------------------------------------------
在target文件夾下將有如下包。
即表示編譯成功完成。
（3）安裝并啟動ranger-admin
修改配置文件
關于數據庫安裝，權限設置等，本文不再展開。
# 進入target目錄
cd /opt/apache-ranger-1.2.0/target/
# 解壓ranger-1.2.0-admin.tar.gz
tar -zxvf ranger-1.2.0-admin.tar.gz
# 進入ranger-1.2.0-admin目錄
cd ranger-1.2.0-admin
# 修改 install.properties
vim install.properties
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java-8.0.17.jar // 修改為準備工作中下載的jar包及路徑
db_root_user=root
db_root_password=123456
db_host=192.168.56.105
db_name=ranger
db_user=rangeradmin
db_password=rangeradmin
# 可以注銷如下內容
#Source for Audit Store. Currently only solr is supported.
# * audit_store is solr
## audit_store=solr
# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits
## audit_solr_urls=
## audit_solr_user=
## audit_solr_password=
## audit_solr_zookeepers=
修改 setup.sh
// 注銷如下內容
## if [ "$?" != "0" ]
## then
## validateDefaultUsersPassword 'admin' "${rangerAdmin_password}"
## validateDefaultUsersPassword 'rangertagsync' "${rangerTagsync_password}"
## validateDefaultUsersPassword 'rangerusersync' "${rangerUsersync_password}"
## validateDefaultUsersPassword 'keyadmin' "${keyadmin_password}"
## fi
安裝
./ setup.sh
注意，如果中途報如下錯誤。
2019-08-20 08:54:22,460 [I] '/usr/local/java/bin/java' command found
setup.sh:行325: bc: 未找到命令
setup.sh: 第 325 行:[: -eq: 期待一元表達式
即缺少bc命令，安裝后即可。
最終出現如下結果：
2019-08-20 09:00:18,240 [I] --------- Verifying Ranger DB connection ---------
2019-08-20 09:00:18,240 [I] Checking connection..
geradmin' -p '********' -noheader -trim -c \; -query "SELECT version();"
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2019-08-20 09:00:18,838 [I] Checking connection passed.
2019-08-20 09:00:19,091 [I] DB FLAVOR :MYSQL
2019-08-20 09:00:19,092 [I] --------- Verifying Ranger DB connection ---------
2019-08-20 09:00:19,092 [I] Checking connection..
geradmin' -p '********' -noheader -trim -c \; -query "SELECT version();"
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2019-08-20 09:00:19,660 [I] Checking connection passed.
Installation of Ranger PolicyManager Web Application is completed.
表示已完成安裝。
（4）啟動ranger-admin
# 進入ews目錄
cd ews
# 啟動服務
./ranger-admin-services.sh start
/** 正常情況下出現如下結果*/
Starting Apache Ranger Admin Service
Apache Ranger Admin Service with pid 236275 has started.
# 查看是否啟動
ps -ef|grep ranger
或
netstat -lntp|grep 6080
（5）web端驗證
啟動完成后，可以用web端登錄驗證，默認端口為6080,默認用戶名密碼均為admin。
登錄后，結果如下：
3、配置hdfs插件
（1）在ranger-admim上配置一個服務
注意配置的服務名與底層配置文件里配置的服務名一致。
（2）修改配置文件
# 解壓
tar -zxvf ranger-1.2.0-hdfs-plugin.tar.gz
# 進入目錄
cd /opt/apache-ranger-1.2.0/target/ranger-1.2.0-hdfs-plugin
# 修改install.properties
vim install.properties
/** 主要修改如下內容*/
# Example:
# POLICY_MGR_URL=http://policymanager.xasecure.net:6080
#
POLICY_MGR_URL=http://192.168.1.17:6080
#
# This is the repository name created within policy manager
#
# Example:
# REPOSITORY_NAME=hadoopdev
#
REPOSITORY_NAME=hadoopdev
#
# Set hadoop home when hadoop program and Ranger HDFS Plugin are not in the
# same path.
#
COMPONENT_INSTALL_DIR_NAME=/opt/software/hadoop/hadoop-3.2.0
（3）啟動插件
./enable-hdfs-plugin.sh
/** 結果如下*/
Custom user and group is available, using custom user and group.
+ Tue Aug 20 18:04:14 CST 2019 : hadoop: lib folder=/opt/software/hadoop/hadoop-3.2.0/share/hadoop/hdfs/lib conf folder=/opt/software/hadoop/hadoop-3.2.0/etc/hadoop
+ Tue Aug 20 18:04:14 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/hdfs-site.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.hdfs-site.xml.20190820-180414 ...
+ Tue Aug 20 18:04:14 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-hdfs-audit.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-hdfs-audit.xml.20190820-180414 ...
+ Tue Aug 20 18:04:15 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-hdfs-security.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-hdfs-security.xml.20190820-180414 ...
+ Tue Aug 20 18:04:15 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-policymgr-ssl.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-policymgr-ssl.xml.20190820-180414 ...
+ Tue Aug 20 18:04:16 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20190820180416 ...
Ranger Plugin for hadoop has been enabled. Please restart hadoop to ensure that changes are effective.
網頁上測試成功后即可。
其他組件的測試和hdfs類似，可自行嘗試。
責任編輯：姜華
來源：
今日頭條
