Simple process monitoring script with email alerting

If you don't have or don't want to install additional software for system/application monitoring (like Nagios, Zabbix, Munin, Big Brother, etc.) you may use this simple script:


#!/bin/bash

MAIL="your@email.address"
PROGRAM="httpd"
HOST=$(uname -n)
DATE=$(date)
TMPFILE=/var/tmp/monitor-$(PROGRAM).lock

OUTPUT=$(ps -ef | grep -c "$PROGRAM")
if [ $OUTPUT -eq 1 ]; then
  if [ -f $TMPFILE ]; then
    echo "Lock file exists"
  else
    echo "$DATE $HOST program \"$PROGRAM\" is not running" | mailx -s "\"$PROGRAM\" is not running on $HOST" $MAIL
    touch $TMPFILE
  fi
fi

In PROGRAM variable put the name of the process that you expect to be running, make sure that the monitoring script name will not contain the same string.
Basically, if the program is running "ps -ef | grep program" will return 2 or more rows (one with the program itself and the second one with "grep program").
Otherwise it will only return one row ("grep program") which will trigger the alert and you will get an email.
By creating TMPFILE script will avoid bothering you again and again about the same issue.
Make sure to remove that file after you restart monitored process.

Once the script is ready save it and add to cron, i.e.:
$ crontab -e
* * * * * /path/to/the/script > /dev/null 2>&1

Using bash auto_completion features

When standard bash completion is not enough try installing bash-completion package from EPEL repository.
# yum install bash-completion

Now log off and on again or run:
# source /etc/bash_completion

You should get more logical hints, try this:
# ifup [TAB][TAB]
eth0 eth0:0 eth0.bak lo

See - it's now suggesting (and completing) interface names instead of files in the current directory. Cool stuff!

Howto to add date and time to bash history

It might be useful to to log time stamp to each bash command that is being executed on server you maintain (even more if you're not the only admin there).
To keep everyone's history with time stamps edit /etc/bashrc and add:
export HISTTIMEFORMAT="%y/%m/%d %T "

for one user only edit ~./bash_profile file and add the same line.

The fields that you may use means:
%y - year
%m - month
%d - day
%T - time
%H - hour
%M - minute
%S - second