Knowledge is Power
(Not original article) sql-metadata is a Python library that uses a tokenized query returned by python-sqlparse and generates query metadata. This metadata can return column and table names from your supplied SQL query. Here are a couple of example from the sql-metadata github readme:...
Want to write a Hadoop program in less than 5 minutes? Get in here for a quick check on how it’s done. We use Python and Hadoop streaming to complete the task.
Below bash command will let you find the version of packages for your python interpreter. Make sure you are running the correct version of python enterpreter. Update: 2020-05-14 for i in pandas numpy sqlalchemy logging logging.handlers datetime sys re os...
Connect to msaccess file from Python and tweak it to emit desired format. import pyodbc conn = pyodbc.connect(r’Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=path where you stored the Access file\file name.accdb;’) cursor = conn.cursor() cursor.execute(‘select * from table name’) for row in...
Simple web app in Python using flask. This app connects to a database and compares the cost of calls, using new connection object each time, vs reusing same connection object HTH from flask import Flask import pyodbc...
Got in a situation where you were asked to extract hive queries and the time they took to execute? Steps On log files run below 2 extracts awk ‘match($0, “^([^ ]+).*Completed executing command\\(queryId=([0-9a-z_-]+)\\); Time taken: (.*)”, a) {print “COMPLETE\t” a[1]...
Source: https://snakebite.readthedocs.io/en/latest/client.html Example: >>> from snakebite.client import Client >>> client = Client(“localhost”, 8020, use_trash=False) >>> for x in client.ls([‘/’]): … print x Warning Many methods return generators, which mean they need to be consumed to execute! Documentation will explicitly...
Python code below, It fetches all contents from 5 urls, general discussion, filter exchange, public filters, stock picks and indicators. Destination dir: /tmp File type: csv #!/usr/bin/env python from lxml import html import requests import csv import sys import...
Housekeeping with mysql Downloading data from wsj using python script here Concatenating daily files # Below command concatenates files and skips the first row of each file $ tail -q -n +2 *.tsv > /tmp/stock_aggr.tsv Creating table in mysql drop...
Below code will pull losers and gainers from wsj.com for a given date range, #!/usr/bin/env python from lxml import html import requests import sys import os import humanize import pandas # from datetime import date, timedelta import datetime import re...
More
