2019-09-16 sponge and other moreutils

GNU coreutils are well-known and loved, especially with pipes (of course!). But what may be slightly less known is the collection of command-line tools called moreutils. As their author says, moreutils is a growing collection of the unix tools that nobody thought to write long ago when unix was young. The collection contains a surprisingly large number of very clever little tools. One of them is sponge, which solves the old problem of pipe-manipulating a file in place. Assume that we have a file called file.txt and we want to only leave the lines containing the letter a in it. We can’t just say

grep a file.txt > file.txt

since grep won’t let us; and if it did, it wouldn’t work, as file.txt would be emptied before grep had a chance of going through it all. (You can check that it does so by saying e.g. grep a file.txt | cat > file.txt).

Now, sponge to the rescue. What it does is reads (“soaks”, hence the name) the whole standard input and starts writing only then. We can now say grep a file.txt | sponge file.txt and everything works as expected.

All that is nice and good, and I’ve known it for some time. More recently, however, I looked into more utils from moreutils, ;-) and it turns out that there are quite a few nice ones.

For instance, ts, which works much like cat, but appends a timestamp to every line it processes. It is a nice tool to produce simple log files. Of course, the actual timestamp format is configurable. You can also tell it to produce incremental timestamps, relative to either the previous one or the moment the program was started. Head to the manpage for more options and details.

Yet another one is chronic. It is intended as a wrapper for commands run by cron. The idea is that if a command returns with the exit status of 0, chronic hides its stdout and stderr, but outputs them normally otherwise. This way you may e.g. run a command in cron in verbose mode but do not have any output if it succeeds.

The last one I’m going to mention is ifne. It is very simple – it just runs the given command provided the standard input is not empty, and pipes it in in that case. (It also has an option to invert the default behavior.) It can be very useful when you have to deal with misbehaved command-line utilities which report something to stdout or stderr instead of having a non-zero exit status.

Go to the moreutils website to see more. I am sure you will find something useful.

CategoryEnglish, CategoryBlog