When you manage servers, it can be useful to take a numbers of action automatically (restart a service, add a firewall rules, kill a process, sending an alert, etc.) if the load average is too high. The problem of a lot of shell script about this, it's if the load average down slowly after the action, the script may go wrong and execute another kill, restart, etc that can be a cause of a service downtime.
The solution is to check 2 states of the load average before running yours commands.
Place to the code :
#!/bin/sh
tsleep=2 # time to wait before 2 checks
llimit=8 # load limit before action
action="/home/dni/action.sh" # script or to execute if necessary
load=`cat /proc/loadavg |awk {'print $1'}|cut -d "." -f1` # The load average now
sleep $tsleep
load2=`cat /proc/loadavg |awk {'print $1'}|cut -d "." -f1` # The load average after tsleep
if test "$load" -ge $llimit
then
if test "$load2" -ge $load
then
$action
else
echo "ok" 1>&2
fi
else
echo "ok" 1>&2
Now you have just to put this script into your crontab each minutes for example and says good night to your server ;). You can also download directly this script from : http://www.christophe-casalegno.com/tools/monitor.sh
-
Christophe Casalegno
https://twitter.com/Brain0verride