Author: robin

0

Setup local Hadoop dev environment on macOS

It is always so convenient to have a local environment for learning and quick testing of a scenario. If you are working in macOS environment looking to learn or setup Hadoop locally then you are in the right place. I...

0

LDAP Tutorial: Add User Entries and Group Entries

Source: https://access.redhat.com/documentation/en-US/Fuse_MQ_Enterprise/7.1/html/Security_Guide/files/LDAP-AddUserEntries.html Overview The basic prerequisite for using LDAP authentication in the broker is to have an X.500 directory server running and configured with a collection of user entries and group entries. For users who are unfamiliar with X.500 directory...

0

LDAP – Apache Directory Studio: A Basic Tutorial

Source: http://krams915.blogspot.com/2011/01/ldap-apache-directory-studio-basic.html In this tutorial we will setup a basic LDAP structure containing users and roles. We will be using the excellent Apache Directory Studio IDE. This tutorial will be the basis for our other Spring LDAP integration tutorials. What is...

0

iTerm shell integration

Important – iTerm shell integration Shell integration should be installed on all remote hosts Connections to non standard ssh port (other than 22) can be setup in ~/.ssh/config. Format below Host www.adhocshare.tk HostName 192.168.80.200 Port 10527   Iterm2 bash prompt...

0

Run a script as root

Source: http://www.cyberciti.biz/tips/shell-root-user-check-script.html Sometime it is necessary to find out if a shell script is being run as root user or not. When user account created a user ID is assigned to each user. BASH shell stores the user ID in $UID...

0

Hive statistics using beeline and expect script

Following expect script uses beeline interface to fetch statistics of tables within a database. Use username and queuename with your environment values. #!/usr/bin/expect -f # hive_statistics, v0.1, 2016-05, [email protected] # Usage: ./hive_statistics [database_name] set _database [lindex $argv 0] if {...

0

Hortonworks Data Platform Installation errata – Missing manual

Pre-requisites Creating service users and databases in MySQL JDBC connector error during ambari-server setup with MySQL MySql connection failing during ambari automated installation   Some useful bash scripts Create database and user on mysql for services like ambari, oozie, hue,...

0

MySQL Tips

Show all users Drop all databases Delete all users in MySQL with specific name Check and duplicate user grants Reset root password (mysql 5.7)   Tip #1: Show all users MariaDB [(none)]> select User,Host from mysql.user order by User, Host; Output...

0

Configure log files on HDP platform

  Kafka Storm Ranger HDFS Zookeeper Oozie Knox Hive & Hive metastore   1. Kafka Kafka currently uses org.apache.log4j.DailyRollingFileAppender, which doesn’t allow us to specify the max backup index, max file size. And by default, rolls every hour creating 24...

0

Insightful Hadoop administration commands

Tip #1 Quick list of operations Where? Example: /var/log/hadoop/hdfs cat hdfs-audit.log | awk ‘{cmds[$9]++}END{for (i in cmds)printf “%s %d\n”,i,cmds[i]}’ Results [user@server hdfs]$ cat hdfs-audit.log | awk ‘{cmds[$9]++}END{for (i in cmds)printf “%s %d\n”,i,cmds[i]}’ cmd=setTimes 52 cmd=listStatus 47422 cmd=create 36932 cmd=getfileinfo 7431182...