Knowledge is Power
At times some directories on hdfs has too many inodes (files and folders) and it is really hard to delete. Some instances also lead to out of memory (OOM) errors such as the following error, INFO retry.RetryInvocationHandler: java.io.IOException: com.google.protobuf.ServiceException: java.lang.OutOfMemoryError:...
Querying hive metastore tables can provide more in depth details on the tables sitting in Hive. This article is a collection of queries that probes Hive metastore configured with mysql to get details like list of transactional tables, etc. More...
Get column details of a table Get column details of a table select name from sys.columns where object_name(object_id)=’table_name’ No related posts.
This is a quick and raw method to setup alerts when the filesystem fill above threshold. Pre-requisites Monitored filesystems should be consistent, meaning available across all nodes passwordless ssh should be setup between the nodes. Node where the alert script...
Below bash command will let you find the version of packages for your python interpreter. Make sure you are running the correct version of python enterpreter. Update: 2020-05-14 for i in pandas numpy sqlalchemy logging logging.handlers datetime sys re os...
Follow below steps to download video from a webpage Step 1: Finding m3u8 URL from webpage Open Chrome Developer tools and click the Network tab Navigate to the page with the video and get it to start playing Filter the...
Check ports connectivity using automation with telnet and timeout commands. Timeout will help us not get blocked for a long time. Adjust the timeout value on case to case basis, # vi ~/check_my_server_hostname.sh timeout 2 bash -c “echo ‘exit’ |...
Basic code snippet to make HTTP calls from SQL server
Connect to msaccess file from Python and tweak it to emit desired format. import pyodbc conn = pyodbc.connect(r’Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=path where you stored the Access file\file name.accdb;’) cursor = conn.cursor() cursor.execute(‘select * from table name’) for row in...
Source: http://mtkcode.blogspot.com/2014/08/parse-json-string-by-sql-script.html SQL Server 2016 and above support JSON_VALUE function to parse JSONs. To process JSONs in older versions add this function to database – https://github.com/khanhmai/Parse-JSON-String-by-SQL-script/blob/master/ParseJSON-FUNCTION.sql Then we can write queries such as below, — Query sample 1 select *...
More
