adhocshare

Knowledge is Power

adhocshare Blog

0

Bigdata / Hadoop

Hadoop security practices

References http://hortonworks.com/hadoop-tutorial/manage-security-policy-hive-hbase-knox-ranger/ http://hortonworks.com/blog/best-practices-in-hdfs-authorization-with-apache-ranger/ http://hortonworks.com/blog/best-practices-for-hive-authorization-using-apache-ranger-in-hdp-2-2/ http://hortonworks.com/blog/author/balajiganesan03/ http://www.slideshare.net/hortonworks/ops-workshop-asrunon20150112 Related posts: Deleting users from Ranger database (mysql) Apache Ranger tips and tid bits Remotely debug hadoop Good looking .hiverc file

0

Programming

Uninstalling npm from macOS

Source: http://stackoverflow.com/questions/11177954/how-do-i-completely-uninstall-node-js-and-reinstall-from-beginning-mac-os-x Removing npm from mac sudo rm -rf /usr/local/{lib/node{,/.npm,_modules},bin,share/man}/{npm*,node*,man1/node*} which is the equivalent of (same as above)… sudo rm -rf /usr/local/bin/npm /usr/local/share/man/man1/node* /usr/local/lib/dtrace/node.d ~/.npm ~/.node-gyp /opt/local/bin/node opt/local/include/node /opt/local/lib/node_modules   or (same as above) broken down… To completely uninstall node +...

0

Bigdata / Hadoop / Mac

Can’t connect Excel to Hive using ODBC driver on MAC

So you done everything right and can’t connect Excel to Hive using ODBC driver on your macOS? Let’s see what is going on. Are you running El Capitan on Sierra? Well I was running Sierra and tried connecting before while...

0

Bigdata / Hadoop

Connecting SQuirrel SQL to Hive

Pre-requisites In order to connect SQuirrel SQL client we need the following prerequisites, Client – http://squirrel-sql.sourceforge.net/ Hive connection JARs (found in lib directories) Hive JDBC JAR – hive-jdbc-1.2.1-standalone.jar Hadoop common JAR (for ) – hadoop-common-2.7.2.jar Running HiveServer2 instance For connections use the following...

0

Bigdata / Hadoop

Creating Hive tables on compressed files

Stuck with creating Hive tables on compressed files? Well the documentation on apache.org suggests that Hive natively supports compressed file – https://cwiki.apache.org/confluence/display/Hive/CompressedStorage Lets try that out. Store a snappy compressed file on HDFS. … thinking, I do not have such file… Wait!...

0

Bigdata / Hadoop

Query escaped JSON string in Hive

There are times when we want to parse a string that is actually a JSON. Usually that could be done with built in functions of Hive such as get_json_object(). Though get_json_object cannot parse JSON Array from my experience. These array...

0

Linux

Setting up tmux without root access

At times we want access to tmux and we are just stuck due to admin restrictions or root access. There are three important parts to setting up tmux, It is dependent on libevent Compiling and installing on non-system folders require customized...

0

Bigdata / Hadoop

Using JSON SerDe in Hive

Using JsonSerDe in Hive Download JSON Serde – https://github.com/rcongiu/Hive-JSON-Serde Compile command for hive 1.2.1 – “mvn -Pcdh5 -Dcdh5.hive.version=1.2.1 clean package” . change hive version per the environment Copy json-serde/target/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar (or similar) to hive/lib Restart hive Sample JSON with test HiveQLs...

0

Linux / Mac

SSH Keep connection alive

Add below lines to file ~/.ssh/config Host * ServerAliveCountMax 3 ServerAliveInterval 10 TCPKeepAlive yes The first line specifies that this applies to all hosts The second and third specify that SSH2 keepalive messages should be sent every 10 seconds during...

0

Mac

Change DNS server on Mac from command line

Fetch DNS servers list networksetup -getdnsservers Wi-Fi Set DNS servers list Format: networksetup -setdnsservers (Network Service) (DNS IP list) networksetup -setdnsservers Wi-Fi 208.67.222.222 208.67.220.220 8.8.8.8 8.8.4.4   References: How to Change DNS from Command Line of Mac OS X   Related posts: MAC terminal...