Hortonworks Data Platform Installation errata – Missing manual
- Pre-requisites
- Creating service users and databases in MySQL
- JDBC connector error during ambari-server setup with MySQL
- MySql connection failing during ambari automated installation
Some useful bash scripts
- Create database and user on mysql for services like ambari, oozie, hue, etc.
1. Pre-requisites
- Setup password-less ssh for root user
- Install some basic packages. Some could be environment specific, for e.g. mysql
yum install -y ntp telnet curl wget unzip mysql-server mysql-connector-java
- On CentOS 6, disable THP
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
- Configure services
chkconfig iptables off chkconfig mysqld on chkconfig ntpd on service mysqld start service ntpd start service iptables stop
- Install Oracle JDKhttp://www.oracle.com/technetwork/java/javase/downloads/index.html
- Ambari.repo format
#VERSION_NUMBER=2.2.2.0-460 [Updates-ambari-2.2.2.0] name=ambari-2.2.2.0 - Updates #baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0 baseurl=http://192.168.56.1:8000/AMBARI-2.2.2.0/centos6/2.2.2.0-460 gpgcheck=0 gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
- If using python http server, fix repo URLs
http://192.168.56.1:8000/AMBARI-2.2.2.0/centos6/2.2.2.0-460 http://192.168.56.1:8000/centos6-hdp-2.3.4.7/HDP/centos6/2.x/updates/2.3.4.7 http://192.168.56.1:8000/centos6-hdp-2.3.4.7/HDP-UTILS-1.1.0.20/repos/centos6
2. Creating service users and databases in MySQL
Automation script in this article below.
CREATE USER '<SERVICEUSER>'@'%' IDENTIFIED BY '<SERVICEPASSWORD>'; GRANT ALL PRIVILEGES ON *.* TO '<SERVICEUSER>'@'%'; CREATE USER '<SERVICEUSER>'@'localhost' IDENTIFIED BY '<SERVICEPASSWORD>'; GRANT ALL PRIVILEGES ON *.* TO '<SERVICEUSER>'@'localhost'; CREATE USER '<SERVICEUSER>'@'<SERVERFQDN>' IDENTIFIED BY '<SERVICEPASSWORD>'; GRANT ALL PRIVILEGES ON *.* TO '<SERVICEUSER>'@'<SERVERFQDN>'; FLUSH PRIVILEGES;
Login with the same user using a password to verify, then create the database
$ mysql -u <SERVICEUSER> -p CREATE DATABASE <SERVICEDATABASE>; USE <SERVICEDATABASE>;
3. JDBC connector error during ambari-server setup with MySQL
ambari-server setup ... ... Enter advanced database configuration [y / n] (n)? Y ================================================== ============================ Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL [4] - PostgreSQL ================================================== ============================ Enter choice (1): 3 Hostname (localhost): Port (3306): Database Name (ambari): Username (ambari): Enter Database Password (bigdata): WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java. Press to continue. ERROR: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java. ERROR: Exiting with exit code -1. REASON: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
To fix the error install mysql-connector-java using yum
yum install -y mysql-connector-java
4. MySql connection failing during ambari automated installation
Check if the jdbc driver has been registered with ambari-server
sudo ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
Some useful bash scripts
Create database and user on mysql for services like ambari, oozie, hue, etc.
usage: sudo ./create_db-2.sh -u userambari -p ambari -d ambari -s hdp2-3-4-7-c7.sandbox
if [[ $EUID -ne 0 ]]; then
echo "This script must be run as root" 1>&2
exit 1
fi
while [[ $# > 1 ]]
do
key="$1"
case $key in
-u|--user)
_SERVICEUSER="$2"
shift # past argument
;;
-p|--password)
_SERVICEPASSWORD="$2"
shift # past argument
;;
-d|--database)
_SERVICEDATABASE="$2"
shift # past argument
;;
-s|--server)
_SERVERFQDN="$2"
shift # past argument
;;
*)
# unknown option
printf "Usage: %s: [-u <user> -p <password> -d <database name> -s <server fqdn> \n" $0
exit 2
;;
esac
shift # past argument or value
done
if [ -z $_SERVICEUSER ] || [ -z $_SERVICEPASSWORD ] || [ -z $_SERVICEDATABASE ] || [ -z $_SERVERFQDN ] ; then
printf "Error: Missing arguments.\nUsage: %s: [-u <user> -p <password> -d <database name> -s <server fqdn>]\n" $0
exit 2
fi
echo -e "### Following commands will be executed in mysql. ###\n"
echo "CREATE DATABASE $_SERVICEDATABASE;"
echo "CREATE USER '$_SERVICEUSER'@'%' IDENTIFIED BY '$_SERVICEPASSWORD';"
echo "GRANT ALL PRIVILEGES ON $_SERVICEDATABASE.* TO '$_SERVICEUSER'@'%';"
echo "CREATE USER '$_SERVICEUSER'@'localhost' IDENTIFIED BY '$_SERVICEPASSWORD';"
echo "GRANT ALL PRIVILEGES ON $_SERVICEDATABASE.* TO '$_SERVICEUSER'@'localhost';"
echo "CREATE USER '$_SERVICEUSER'@'$_SERVERFQDN' IDENTIFIED BY '$_SERVICEPASSWORD';"
echo "GRANT ALL PRIVILEGES ON $_SERVICEDATABASE.* TO '$_SERVICEUSER'@'$_SERVERFQDN';"
echo -e "FLUSH PRIVILEGES;\n"
read -p "### Continue (y/n)? " answer
case ${answer:0:1} in
y|Y )
eval $(echo mysql -u root -e \""CREATE DATABASE $_SERVICEDATABASE;\"")
eval $(echo mysql -u root -e \""CREATE USER '$_SERVICEUSER'@'%' IDENTIFIED BY '$_SERVICEPASSWORD';\"")
eval $(echo mysql -u root -e \""GRANT ALL PRIVILEGES ON $_SERVICEDATABASE.* TO '$_SERVICEUSER'@'%';\"")
eval $(echo mysql -u root -e \""CREATE USER '$_SERVICEUSER'@'localhost' IDENTIFIED BY '$_SERVICEPASSWORD';\"")
eval $(echo mysql -u root -e \""GRANT ALL PRIVILEGES ON $_SERVICEDATABASE.* TO '$_SERVICEUSER'@'localhost';\"")
eval $(echo mysql -u root -e \""CREATE USER '$_SERVICEUSER'@'$_SERVERFQDN' IDENTIFIED BY '$_SERVICEPASSWORD';\"")
eval $(echo mysql -u root -e \""GRANT ALL PRIVILEGES ON $_SERVICEDATABASE.* TO '$_SERVICEUSER'@'$_SERVERFQDN';\"")
eval $(echo mysql -u root -e \""FLUSH PRIVILEGES;\"")
;;
* )
echo Exiting.
;;
esac
