adhocshare Blog

0

snakebite – Python HDFS client

Source: https://snakebite.readthedocs.io/en/latest/client.html Example: >>> from snakebite.client import Client >>> client = Client(“localhost”, 8020, use_trash=False) >>> for x in client.ls([‘/’]): … print x   Warning Many methods return generators, which mean they need to be consumed to execute! Documentation will explicitly...

0

Download youtube playlist

Setup sudo apt-get install ffmpeg brew install youtube-dl Command youtube-dl –extract-audio –audio-format mp3 -o “%(title)s.%(ext)s” <url to playlist>   References https://askubuntu.com/questions/564567/how-to-download-playlist-from-youtube-dl Related posts: Compiling in $HOME Installing perl modules in non standard directory Bulk add remote host fingerprint in ~/.ssh/known_hosts...

0

Better WSL terminal

In your search of a Linux terminal on Windows you might have stumbled across projects such as cmder and WSL. There are many more projects like those. Personally for me this was the only configuration (in this post) that ended...

0

Hadoop like a PRO

Lookup yarn queue of a user from bash This bash function will lookup capacity scheduler XML and return queues for the user getYarnQueue() { grep $1 -B 1 /etc/hadoop/conf/capacity-scheduler.xml | awk -F’.’ ‘/name/{print $(NF-1)}’ } Works on Hortonworks HDP. Usage:...

0

Java unsupported major minor version 52.0

If you are reading this then perhaps you are having java version mismatch. Several apps based on implementations may present you with errors such as class not found error (yes! That’s what R did to me) or class version error –...

0

Merge multiple consecutive lines

At times there is a need to merge multiple consecutive lines to one. paste command makes it very easy to merge lines. Syntax: paste -d’,’ – – < input_file See the example below, robin@Home-XPS:~$ echo -e ‘1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12’ 1 2 3...

0

Chocolatey package manager for Windows

You will find a few similarities with Homebrew (https://brew.sh/) and linuxbrew (http://linuxbrew.sh/). Chocolatey (https://chocolatey.org/) is a similar package manager for windows.   Installation Installation command is a little weird than other OS. Inside an administrative command prompt, type below >”%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe” -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command “iex ((New-Object System.Net.WebClient).DownloadString(‘https://chocolatey.org/install.ps1’))” && SET “PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin”...

0

Etrade Pro Strategy Scanner

WN=Under+$5,+High+Vol+High+Volatility&Sh_CMU=on&Sh_BBU=on&Sh_SV=on&QSV=10&Sh_NR7_1=on&MaxPrice=5&MinVol3M=500000&MinRV=1.5&MinVWVP=0.3&MinRange120P=2&MinRange5DP=3&MinFCD=0.01&MinBB=1.5&X_NYSE=on&X_ARCA=on&X_AMEX=on&XN=on   The objective Use the above etrade scanner for finding, Stocks NOT on the OTCBB or Pinks, US big board stocks only, SP between $.01 and $5.00. Bullish stocks having high volume and volatility This is good for taking...

0

Hive msck repair not working

Okay, so msck repair is not working and you saw something as below, 0: jdbc:hive2://hive_server:10000> msck repair table mytable; Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) It seems to appear because of higher...