Knowledge is Power
Got in a situation where you were asked to extract hive queries and the time they took to execute? Steps On log files run below 2 extracts awk ‘match($0, “^([^ ]+).*Completed executing command\\(queryId=([0-9a-z_-]+)\\); Time taken: (.*)”, a) {print “COMPLETE\t” a[1]...
Below is data storage estimator based on message size and throughput. Input HDFS replication and the amount of data end users will generate over time using raw data. Hope this helps you. Data calculator Message size in bytes Message per...
Source: https://unix.stackexchange.com/questions/199863/create-many-files-with-random-content For example, Name size file1.01 2K file2.02 3K file3.03 5K etc. Strategy – 1 #! /bin/bash for n in {1..1000}; do dd if=/dev/urandom of=file$( printf %03d “$n” ).bin bs=1 count=$(( RANDOM + 1024 )) done ...
Source: https://snakebite.readthedocs.io/en/latest/client.html Example: >>> from snakebite.client import Client >>> client = Client(“localhost”, 8020, use_trash=False) >>> for x in client.ls([‘/’]): … print x Warning Many methods return generators, which mean they need to be consumed to execute! Documentation will explicitly...
Impatient? Each AP can have same SSID They should be on different non-overlapping channels Configuration tips Set radio power Set radio power to as low or medium Use RSSI in advanced settings to facilitate clients roaming to the “best” AP....
Setup sudo apt-get install ffmpeg brew install youtube-dl Command youtube-dl –extract-audio –audio-format mp3 -o “%(title)s.%(ext)s” <url to playlist> References https://askubuntu.com/questions/564567/how-to-download-playlist-from-youtube-dl Related posts: Compiling in $HOME Installing perl modules in non standard directory Bulk add remote host fingerprint in ~/.ssh/known_hosts...
In your search of a Linux terminal on Windows you might have stumbled across projects such as cmder and WSL. There are many more projects like those. Personally for me this was the only configuration (in this post) that ended...
Lookup yarn queue of a user from bash This bash function will lookup capacity scheduler XML and return queues for the user getYarnQueue() { grep $1 -B 1 /etc/hadoop/conf/capacity-scheduler.xml | awk -F’.’ ‘/name/{print $(NF-1)}’ } Works on Hortonworks HDP. Usage:...
If you are reading this then perhaps you are having java version mismatch. Several apps based on implementations may present you with errors such as class not found error (yes! That’s what R did to me) or class version error –...
At times there is a need to merge multiple consecutive lines to one. paste command makes it very easy to merge lines. Syntax: paste -d’,’ – – < input_file See the example below, robin@Home-XPS:~$ echo -e ‘1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12’ 1 2 3...
More
