Ambari is completely open source management platform for provisioning, managing, monitoring and securing Apache Hadoop clusters. Apache Ambari takes the guesswork out of operating Hadoop. As part of the Hortonworks Data Platform, allows enterprises to plan, install and securely configure HDP making it easier to provide ongoing cluster maintenance and management, no matter the size of the cluster.
I've total 6 nodes, I'll use one node for Ambari Server and remaining nodes for Hadoop Cluster.
EN01 ==> Ambari Server
NN01 ==> Name Node
DN01 ==> Data Node 1
DN02 ==> Data Node 2
DN03 ==> Data Node 3
DN04 ==> Hadoop Server components + Clients
Apache Ambari Installation
Download Ambari from below location
https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-installation/content/download_the_ambari_repo_lnx7.html
[hdpsysuser@en01 ~]$ su -
Password:
Last login: Mon Jul 17 16:01:30 AST 2017 on pts/0
[root@nn01 ~]# pwd
/root
[root@en01 ~]# wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/ambari.repo -O /etc/yum.repos.d/ambari.repo
2017-07-25 11:11:00 URL:http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/ambari.repo [304/304] -> "/etc/yum.repos.d/ambari.repo" [1]
[root@nn01 ~]# yum repolist
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
ambari-2.5.0.3 | 2.9 kB 00:00:00
ambari-2.5.0.3/primary_db | 8.5 kB 00:00:00
repo id repo name status
ambari-2.5.0.3 ambari Version - ambari-2.5.0.3 12
repolist: 12
[root@en01 ~]# yum install ambari-server
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.x86_64 0:2.5.0.3-7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================
Installing:
ambari-server x86_64 2.5.0.3-7 ambari-2.5.0.3 663 M
Transaction Summary
=================================================================================================================================================
Install 1 Package
Total download size: 663 M
Installed size: 722 M
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7Server/ambari-2.5.0.3/packages/ambari-server-2.5.0.3-7.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY
Public key for ambari-server-2.5.0.3-7.x86_64.rpm is not installed
ambari-server-2.5.0.3-7.x86_64.rpm | 663 MB 00:17:14
Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Importing GPG key 0x07513CAD:
Userid : "Jenkins (HDP Builds) <jenkin@hortonworks.com>"
Fingerprint: df52 ed4f 7a3a 5882 c099 4c66 b973 3a7a 0751 3cad
From : http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
Installing : ambari-server-2.5.0.3-7.x86_64 1/1
Verifying : ambari-server-2.5.0.3-7.x86_64 1/1
Installed:
ambari-server.x86_64 0:2.5.0.3-7
Complete!
[root@nn01 ~]#
Set Up the Ambari Server
[root@en01 ~]# ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/default
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? n
Configuring database...
Default properties detected. Using built-in database.
Configuring ambari database...
Checking PostgreSQL...
Running initdb: This may take up to a minute.
Initializing database ... OK
About to start PostgreSQL
Configuring local database...
Configuring PostgreSQL...
Restarting PostgreSQL
Creating schema and user...
done.
Creating tables...
done.
Extracting system views...
ambari-admin-2.5.0.3.7.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
1- Login to Ambari server
By default Ambari listens on 8080, so if this port is already used stop the related service first and then runt the Ambari server.
[root@en01 ~]# cd /usr/hadoopsw/prestoadmin/presto-admin-install/bin/
[root@en01 bin]# ./presto-admin server stop
[localhost] out: Stopping presto
[localhost] out: Stopped 8255
[localhost] out:
[root@en01 bin]# ambari-server status
Using python /usr/bin/python
Ambari-server status
Ambari Server not running. Stale PID File at: /var/run/ambari-server/ambari-server.pid
[root@en01 bin]# ambari-server start
Using python /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start...............
Server started listening on 8080
DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.
[root@en01 bin]# ambari-server status
Using python /usr/bin/python
Ambari-server status
Ambari Server running
Found Ambari Server PID: 4126 at: /var/run/ambari-server/ambari-server.pid
2- Now log in to Ambari Web using a web browser, default user name/password: admin/admin. For a new cluster, the Cluster Install wizard displays a Welcome page. choose Launch Install Wizard.
http://<<yourServer>>:8080/#/login
http://en01:8080/#/login
Uninstall Ambari
Follow the instructions
For the next steps you need to run the below commands' output to be provided in the Ambari wizard.
[root@en01 ~]# hostname --fqdn
nn01
[root@en01 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
70:30:60:31:6f:20:14:21:26:d5:3b:a3:e7:fd:c3:f0 root@nn01
The key's randomart image is:
+--[ RSA 2048]----+
|o+*+*oo |
|o. o.+ o |
| .+ . |
| +. o |
| . o S |
| . . . |
| o . + |
| . . E |
| ... |
+-----------------+
[root@en01 ~]# ssh-copy-id root@dn01
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@dn01's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@dn01'"
and check to make sure that only the key(s) you wanted were added.
[root@en01 ~]# ssh root@dn01
Last login: Mon Jul 24 16:40:10 2017
[root@en01 ~]# ssh-copy-id root@nn01
[root@en01 ~]# ssh-copy-id root@dn02
[root@en01 ~]# ssh-copy-id root@dn03
[root@en01 ~]# ssh root@dn02
[root@en01 ~]# ssh root@dn03
[root@en01 ~]# ssh root@dn04
[root@en01 ~]# cat /root/.ssh/id_rsa
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAyNIU01WyjpZDMZaQ4ZA/oR1NVL7RsgJWU8N2colXOsDkJtG3
812Yh7XNU9kQxH1XFDVQ1J1a+rD2yxXyOjyiVFHFTq6op+BN6iZcQIEZnmf0/Sz1
...
...
-----END RSA PRIVATE KEY-----
ntpd should be running on all hosts, time should be synchronized with NTP server
[root@en01 ~]# ntpdate -u 192.168.0.90
[root@dn01 ~]# service ntpd start
[root@nn01 ~]# ntpdate -u 192.168.0.90
[root@dn01 ~]# ntpdate -u 192.168.0.90
[root@dn02 ~]# ntpdate -u 192.168.0.90
[root@dn03 ~]# ntpdate -u 192.168.0.90
[root@dn04 ~]# ntpdate -u 192.168.0.90
If you face any host check issues resolve them
[root@dn03 ~]# ps -eaf | grep java
[hdpsysuser@dn01 ~]$ w
16:38:03 up 84 days, 1:10, 3 users, load average: 0.17, 0.06, 0.06
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/0 dn04 16:37 41.00s 8.54s 3.23s /usr/bin/python /usr/libexec/urlgrabber-ext-down
hdpsysus pts/1 ws9374.elm.com.s 16:37 3.00s 0.16s 0.01s w
For Hive below MySQL user used.
mysql> create database hive;
mysql> CREATE USER 'hive'@'en01' IDENTIFIED BY 'P@ssw0rd';
mysql> GRANT all on *.* to 'hive'@'%' identified by 'P@ssw0rd' with grant option;
mysql> flush privileges;
[root@en01 ~]# mysql -u hive -p
Setup ambari to use MySQL JDBC Driver
[root@en01 ~]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/hadoopsw/apache-hive-2.1.1-bin/lib/mysql-connector-java-5.1.42-bin.jar
Using python /usr/bin/python
Setup ambari-server
Copying /usr/hadoopsw/apache-hive-2.1.1-bin/lib/mysql-connector-java-5.1.42-bin.jar to /var/lib/ambari-server/resources
If you are updating existing jdbc driver jar for mysql with mysql-connector-java-5.1.42-bin.jar. Please remove the old driver jar, from all hosts. Restarting services that need the driver, will automatically copy the new jar to the hosts.
JDBC driver was successfully initialized.
Ambari Server 'setup' completed successfully.
Install Required RH Packages (if required)
I had to install the below packages
subscription-manager register
Username: xxxxxxxx
Password:xxxxxxx
subscription-manager repos
subscription-manager list --available --all
subscription-manager attach --pool=8a85f98156fa00f40156fa51506e4415
subscription-manager repos
yum install redhat-lsb-core
rpm -qa | grep redhat-lsb
subscription-manager unregister
[root@nn01 ~]# yum repolist
[root@nn01 ~]# yum-config-manager --enable rhel-7-server-optional-rpms
[root@nn01 ~]# yum install libtirpc-devel
[root@dn01 ~]# yum install python-devel
If your installation goes with something wrong, you may need to uninsall Ambari and reinstall.
[root@dn04 ~]# ps aux | grep ambari-server
root 4126 0.4 2.9 10520524 713096 ? Sl Jul24 5:38 /usr/java/default/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -Dsun.zip.disableMemoryMapping=true -Xms512m -Xmx2048m -XX:MaxPermSize=128m -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -cp /etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar org.apache.ambari.server.controller.AmbariServer
root 8841 0.0 0.0 112648 960 pts/0 S+ 11:16 0:00 grep --color=auto ambari-server
[root@dn04 ~]# rpm -qa | grep ambari
ambari-server-2.5.0.3-7.x86_64
[root@dn04 ~]# ambari-server stop
Using python /usr/bin/python
Stopping ambari-server
Waiting for server stop...
Ambari Server stopped
[root@dn04 ~]# ambari-server reset
Using python /usr/bin/python
Resetting ambari-server
**** WARNING **** You are about to reset and clear the Ambari Server database. This will remove all cluster host and configuration information from the database. You will be required to re-configure the Ambari server and re-run the cluster wizard.
Are you SURE you want to perform the reset [yes/no] (no)? yes
Confirm server reset [yes/no](no)? yes
Resetting the Server database...
Creating schema and user...
done.
Creating tables...
done.
Ambari Server 'reset' completed successfully.
[root@dn04 ~]# ambari-agent stop
[root@dn04 ~]# yum erase ambari-server
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.x86_64 0:2.5.0.3-7 will be erased
--> Finished Dependency Resolution
ambari-2.5.0.3 | 2.9 kB 00:00:00
https://www.apache.org/dist/cassandra/redhat/311x/repodata/repomd.xml: [Errno 12] Timeout on https://www.apache.org/dist/cassandra/redhat/311x/repodata/repomd.xml: (28, 'Operation timed out after 30001 milliseconds with 0 out of 0 bytes received')
Trying other mirror.
https://www.apache.org/dist/cassandra/redhat/311x/repodata/repomd.xml: [Errno 12] Timeout on https://www.apache.org/dist/cassandra/redhat/311x/repodata/repomd.xml: (28, 'Operation timed out after 30001 milliseconds with 0 out of 0 bytes received')
Trying other mirror.
cassandra/signature | 819 B 00:00:00
cassandra/signature | 2.9 kB 00:00:00 !!!
mysql-connectors-community/x86_64 | 2.5 kB 00:00:00
mysql-connectors-community/x86_64/primary_db | 16 kB 00:00:00
mysql-tools-community/x86_64 | 2.5 kB 00:00:00
mysql57-community/x86_64 | 2.5 kB 00:00:00
Not using downloaded repomd.xml because it is older than what we have:
Current : Sun Jul 16 12:16:47 2017
Downloaded: Mon Apr 10 14:10:29 2017
treasuredata/7Server/x86_64 | 2.9 kB 00:00:00
Dependencies Resolved
=================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================
Removing:
ambari-server x86_64 2.5.0.3-7 @ambari-2.5.0.3 722 M
Transaction Summary
=================================================================================================================================================
Remove 1 Package
Installed size: 722 M
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Note: This output shows SysV services only and does not include native
systemd services. SysV configuration data might be overridden by native
systemd configuration.
If you want to list systemd services use 'systemctl list-unit-files'.
To see services enabled on particular target use
'systemctl list-dependencies [target]'.
ambari-server 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Erasing : ambari-server-2.5.0.3-7.x86_64 1/1
warning: file /usr/lib/python2.6/site-packages/ambari_server/utils.pyo: remove failed: No such file or directory
...
...
Verifying : ambari-server-2.5.0.3-7.x86_64 1/1
Removed:
ambari-server.x86_64 0:2.5.0.3-7
Complete!
[root@dn04 ~]# rm -rf /var/lib/ambari-server
[root@dn04 ~]# rm -rf /var/run/ambari-server
[root@dn04 ~]# rm -rf /usr/lib/ambari-server
[root@dn04 ~]# rm -rf /etc/ambari-server
[root@dn04 ~]# rm -rf /var/log/ambari-server
[root@dn04 ~]# rm -rf /usr/lib/python2.6/site-packages/ambari*
[root@dn04 ~]# ambari-agent stop
[root@dn04 ~]# yum erase ambari-agent
[root@dn04 ~]# rm -rf /var/lib/ambari-agent
[root@dn04 ~]# rm -rf /var/run/ambari-agent
[root@dn04 ~]# rm -rf /usr/lib/amrbari-agent
[root@dn04 ~]# rm -rf /etc/ambari-agent
[root@dn04 ~]# rm -rf /var/log/ambari-agent
rm -rf /var/lib/ambari-agent
rm -rf /var/run/ambari-agent
rm -rf /usr/lib/amrbari-agent
rm -rf /etc/ambari-agent
rm -rf /var/log/ambari-agent
Reset Ambari Agent
[root@dn03 ~]# ambari-agent stop
[root@dn03 ~]# ambari-agent reset nn01
[root@dn03 ~]# ambari-agent start
Binaries Information
[root@dn04 ~]# ls -l /usr/hdp/current
[root@dn04 ~]# ls -l /usr/hdp
Configs for each component
[root@dn04 ~]# ls -l /etc
No comments:
Post a Comment