My molb home

Small collection of (maybe) useful tools

All programs are hosted on github!

Unix tips

Random tips and tricks for shell users.

uniq without loosing line order

Sometimes you want to use uniq, but retain the order of the processed lines, e.g.:

original input after sort -u desired output
foo
foo
bar
bar
foo
foo
bar

Here is a shell one-liner doing this:

nl -ba | sort -suk2 | sort -n | cut -f2-

or this one for uniq options:

nl -ba | sort -sk2 | uniq $YOUR_OPTIONS -f1 | sort -n | cut -f2-

Using a short and not representative benchmark on a 75 MiB file shows a slowdown of roughly 45% compared to regular sort -u. For a way faster solution with instant feedback check out this short python script.

Everlasting bash history

This really makes life easier! Append the lines below to your ~/.bashrc to have an (almost) forever lasting history of your shell constructs. Also, history is written right after every command, which is good because:

From time to time, reducing the size of the history (e.g. with stable uniq above) might be useful.

shopt -s histappend
export HISTCONTROL=ignoredups:ignorespace
export HISTFILESIZE=1000000
export HISTSIZE=1000000
export R_HISTSIZE=1000000
if [ "$PROMPT_COMMAND" ] ; then
  PROMPT_COMMAND="$PROMPT_COMMAND;history -a"
else
  PROMPT_COMMAND="history -a"
fi

Automatically sort mailing list messages via maildrop

This mailfilter excerpt automatically sorts mailing list messages into mlists/ML_NAME, even creating the folder when it does not exist! Saves you the tedious task of writing rules for all subscribed lists...

if (! /^X-List-Administrivia:\s*yes/ && (/^(?:X-BeenThere|X-Mailing-List):\s*(.*)/ || /^List-Post:\s*/ || /^Mailing-List:\s*list\s*([^;]*)/))
{
    if (getaddr($MATCH1) =~ /^[_A-Za-z0-9-]+/)
    {
        mlname=tolower($MATCH)
        mldir="$MAILDIR/.mlists.$mlname"
        `test -e "$mldir"`
        if ($RETURNCODE != 0)
            `maildirmake "$mldir"`
        to "$mldir"
    }
}