Insightful Hadoop administration commands
Tip #1
Quick list of operations
Where?
Example: /var/log/hadoop/hdfs
cat hdfs-audit.log | awk '{cmds[$9]++}END{for (i in cmds)printf "%s %d\n",i,cmds[i]}'
Results
[user@server hdfs]$ cat hdfs-audit.log | awk '{cmds[$9]++}END{for (i in cmds)printf "%s %d\n",i,cmds[i]}' cmd=setTimes 52 cmd=listStatus 47422 cmd=create 36932 cmd=getfileinfo 7431182 cmd=mkdirs 3484 cmd=setPermission 6353 cmd=delete 1660 cmd=rename 30814 cmd=open 14287952 oozie 589 hue 25 cmd=getEZForPath 119 cmd=contentSummary 699 cmd=setReplication 1386 hive 19186