Tag: sed

Best ever Linux/Unix tools

So as you might have guessed I’m a Unix/Linux guy. As such you love certain tools built into both. My favorite editor is vi/vim, and text manipulation with grep, awk and sed is just short of awesome. For example I’m writing a script that checks to see if EMC’s avtar.bin is running and then to grab it’s process ID or PID and then do the nice -n 19 on it. It’ll require a marker of sorts that records date etc.

But it incorporates using the awk utility. With that you can pick out elements of text separated by white space among many other things it can do. Try that in Windows.

Now grep is a searching tool too, if you’re looking for said avtar.bin you just do a ps -ef|grep avtar.bin and you can get all the processes that are avtar.bin.

Now sed is short for stream editor. With it you can essentially change text in a file on the fly using pattern matching. That’s the other powerful thing about Unix and Linux. You can do some really amazingly complex stuff in it just using the tools I’ve mentioned.

The accomplishment for the week (Tech!)

So where I work – instead of doing standard mysqldump SQL dumps – they copy the INNODB files into a backup. These backups an beĀ  6GB or more per week. So it was killing the VM the MySQL server runs on.

Now my predecessor had started writing a prune-mysql-backups.sh script. All he’d done was do a list files (ls -At) in time order and then create a text file called filelist.

But the thing was, the dated files went in ascending order. I didn’t want them that way. And apparently he didn’t know enough about sort and sed to complete the script. Sort or sort does exactly what it says, sorts a list of items. The term sed means Stream Editor.

So I had to fire sort -r which means in reverse chronological order then I ran sed to strip out the three newer items in the list, and then pass that file in to a section that walked through the list and rm -rf’d the directories.

That last part ‘rm -rf’ is a goodie. In Unix parlance rm means REMOVE and the -r means recursive the f being all files. You never, ever, ever want to do this on the root partition which would be ‘rm -rf /’. A fast and hard rule to live by is something has to live in FRONT of the slash, e.g. ‘rm -rf 2013-09-30_21-00-34/ ‘would be ok since it’s going to try to walk in that directory and then delete the content and directory.

Another trick is to alias rm -rf and have it make a confirmation of “Do you really want to do this?” before proceeding. That’s the power of Unix/Linux. You can write scripts and aliases to do pretty much anything.