Author: robin

0

Query escaped JSON string in Hive

There are times when we want to parse a string that is actually a JSON. Usually that could be done with built in functions of Hive such as get_json_object(). Though get_json_object cannot parse JSON Array from my experience. These array...

0

Setting up tmux without root access

At times we want access to tmux and we are just stuck due to admin restrictions or root access. There are three important parts to setting up tmux, It is dependent on libevent Compiling and installing on non-system folders require customized...

0

Using JSON SerDe in Hive

Using JsonSerDe in Hive Download JSON Serde – https://github.com/rcongiu/Hive-JSON-Serde Compile command for hive 1.2.1 – “mvn -Pcdh5 -Dcdh5.hive.version=1.2.1 clean package” . change hive version per the environment Copy json-serde/target/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar (or similar) to hive/lib Restart hive Sample JSON with test HiveQLs...

0

SSH Keep connection alive

Add below lines to file ~/.ssh/config Host * ServerAliveCountMax 3 ServerAliveInterval 10 TCPKeepAlive yes The first line specifies that this applies to all hosts The second and third specify that SSH2 keepalive messages should be sent every 10 seconds during...

0

Change DNS server on Mac from command line

Fetch DNS servers list networksetup -getdnsservers Wi-Fi Set DNS servers list Format: networksetup -setdnsservers (Network Service) (DNS IP list) networksetup -setdnsservers Wi-Fi 208.67.222.222 208.67.220.220 8.8.8.8 8.8.4.4   References: How to Change DNS from Command Line of Mac OS X   Related posts: MAC terminal...

0

SSH Auto completion on OSX

Something that is very much desired while working on shell. I love to have this feature of $ssh ser[TAB] Using brew is my preferred way, just type in – $brew install bash-completion Do not forget to add the text in...

0

HDFS disk consumption – Find what is taking hdfs space

Source: https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.html Script #!/usr/bin/env bash max_depth=5 largest_root_dirs=$(hdfs dfs -du -s ‘/*’ | sort -nr | perl -ane ‘print “$F[1] “‘) printf “%15s %s\n” “bytes” “directory” for ld in $largest_root_dirs; do printf “%15.0f %s\n” $(hdfs dfs -du -s $ld| cut -d’ ‘...

0

WordPress – allow password protected content in search results

Source: https://wordpress.org/support/topic/how-to-allow-password-protected-content-in-search-results SOLUTION: Add this code to your functions.php file and you will get it to work. add_filter( ‘posts_search’, ‘include_password_posts_in_search’ ); function include_password_posts_in_search( $search ) { global $wpdb; $pattern = ” AND ({$wpdb->prefix}posts.post_password = ”)”; $search = str_replace( $pattern, ”,...

0

Use SSH Tunneling to access Ambari web UI, ResourceManager, JobHistory, NameNode, Oozie, and other web UI’s

Source: https://azure.microsoft.com/en-us/documentation/articles/hdinsight-linux-ambari-ssh-tunnel/ Original Author: Larry Franks Excerpts ssh tunnel command ssh -C2qTnNf -D 9876 user-name@machine-name This creates a connection that routes traffic to local port 9876 to the cluster over SSH. The options are: D 9876 – The local port that will route...

0

Disable THP on CentOS 6

For the impatient Disable Transparent Huge Pages on CentOS 6 During Runtime echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag   Surviving reboot sudo vi /etc/init.d/disable-transparent-hugepages #!/bin/sh ### BEGIN INIT INFO # Provides: disable-transparent-hugepages # Required-Start: $local_fs # Required-Stop: # X-Start-Before:...