1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

vit -- Vital Statistics Monitoring Command

Discussion in 'Tomato Firmware' started by Planiwa, Nov 14, 2009.

  1. Planiwa

    Planiwa LI Guru Member

    You may want to copy this to a file called vit, then make it executable with

    Code:
    chmod + x vit
    
    and then you can run it from the command line or from the Scheduler.

    For very close monitoring, you might want to do this from a terminal:

    Code:
    vit -d -s 15
    
    This will run vit every 15 seconds.

    This may take a lot of processing resources.
    Vit reads the entire Conntrack Table, so it should not be run more often than once every 15 seconds, and not with significantly more than 1000 Connections.

    To get help, run

    Code:
    vit -h
    
    If your router keeps crashing, this will help you measure the vital signs before the crash.


    Here it is:

    Code:
    # vit: Vital Stats # Planiwa 2009-11-27
    
    INTERFACES="ppp0 eth1"; HALF=500; DIVI=1000; COUNTWHAT="bytes" # Defaults
    while [ $# -gt 0 ]
    do
    	case "$1" in
    	-*elp|-\?|-h) echo "Syntax		vit [-h] [-t] [-k] [-p] [-d] [-dir] [-all] [Interface ...]
    -h		This help
    -t		Show title headers
    -k		Show kiloBytes instead of MegaBytes, or kiloPackets instead of Packets
    -p		Packets instead of Bytes
    -d		Prepend Datestamp
    -s SLEEPSEC	Repeat every SLEEPSEC seconds, until interrupted
    -l              Send output to syslog
    -dir DIR	Write output to file HHMM in directory DIR.
    		NB: Use CRU in INIT script to control exact Minute!  (Scheduler is unpredictable.)
    		eg: cru a Vit '01,16,31,46 * * * * /jffs/vit -d -p -dir /jffs/vitlog'
    -all		Show all Interfaces
    Interface ...	one or more interfaces.  Default: $INTERFACES
    " ;exit;;
    	-t) echo "LANClients Connections TotMem-Free Load Processes FWUptime WANUptime SysUptime Interface:Tot-Up ...";;
        -k) HALF=0; DIVI=1; KILO="kilo";; ### meaning of kilo is reversed for packets and bytes
        -p) COUNTWHAT="packets";;
        -all) INTERFACES="$(ifconfig|awk '/Link/{print$1}')";;
        -dir) DIR=$2; mkdir -p $DIR; HHMM=$(date '+%H%M'); shift;; ### write to file HHMM in $DIR
        -d) DATESTAMP=Y;;
        -l) LOGGER=Y;;
        -s) SLEEPSEC=$2;shift;;
        -s*) SLEEPSEC=${1#-s};;
    	-*) echo "vit: unrecognized paramter: $1"; set -- SHIFT -h;;
        *)  INTERFACES="$*"; break ;;
        esac
        shift
    done
    case $KILO$COUNTWHAT in
    kilopackets) HALF=500; DIVI=1000;;
    packets) HALF=0; DIVI=1;;
    esac
    
    while :;do
    if [ "$DATESTAMP" != "" ];then DATESTAMP="$(date '+%y%m%d_%H:%M:%S') "; fi
    
    set -- $(cat /proc/meminfo); set -- $((${10}+${13})) ${10}; F=${1%???}-${2%???} # kB
    #F="$((${22}+${31}))-${22}"KiB
    
    if [ -r /proc/sys/net/ipv4/ip_conntrack_count ]
    then CON=$(cat /proc/sys/net/ipv4/ip_conntrack_count) ### Thanks, Teddy Bear!
    else CON=$(cat /proc/net/ip_conntrack | wc -l)        ### This is very costly
    fi
    
    USE=$(arp | wc -l) # set -- $(wl assoclist); USE=$(($#/2))
    
    set -- $(cat /proc/loadavg)
    P=$4
    LOADL=${1%.*}; LOADL=${LOADL#0}; LOADR=${1#*.}; LOADR=${LOADR#0};
    LOADR=$(((LOADR+5)/10)); case $LOADR in 10)LOADR=9;;esac; LOAD=$LOADL.$LOADR
    
    X="";Y="";Z=""; for IF in $INTERFACES;do
      set -- $(ifconfig $IF |grep "$COUNTWHAT:")
      ### Beware: counters are reset at 4.3 GB (mod 2^32) ###
      case $# in
      0) Z="$Z $IF:Down";;
      *) case $COUNTWHAT in
         bytes)   RX=$2; TX=$6;;
         packets) RX=$2; TX=$8;;
         esac
         case $IF in ## these need some work, especially with unusual hardware or setups such as MLPPP
         ppp0|eth0|vlan1) X="$X $IF ${RX#*:} ${TX#*:}";; ## Up from router
         eth1|br0|*)      X="$X $IF ${TX#*:} ${RX#*:}";; ## Down from router
         esac;;
      esac
    done
    set -- $X; for X in $*;do
      case $X in
      [0-9]*) case $KILO$COUNTWHAT in
              *bytes)      if [ ${#X} -le 3 ];then X=0;else X=${X%???};fi; Y="$Y $(((X+$HALF)/$DIVI))";;
              *packets) Y="$Y $(((X+$HALF)/$DIVI))";;
              esac;;
      *)      Y="$Y $X";;
      esac
    done
    set -- $Y
    while [ $# -ge 3 ];do
      Z="$Z $1:$(($2+$3))-$3"
      shift 3
    done
    
    NOWSEC=$(date '+%s')
    
    BOOSEC=$(cat /proc/stat|grep btime); BOOSEC=${BOOSEC#* }
    BOOSECS=$((NOWSEC-BOOSEC))
    D=$((BOOSECS/86400));if [ $D -ge 2 ];then BOOTIME=${D}d;else
    H=$((BOOSECS/3600));if [ $H -ge 2 ];then BOOTIME=${H}h;else
    M=$((BOOSECS/60));if [ $M -ge 2 ];then BOOTIME=${M}m;else
    BOOTIME=${BOOSECS}s;fi;fi;fi
    #BOODATE=$(cat /proc/stat|grep btime|awk '{print strftime("%m%d.%H:%M",$2)}')
    
    WANSEC=$(date -r /var/lib/misc/wantime '+%s')
    if [ $WANSEC -lt 999999 ];then WANSECS=$WANSEC;WANPRE='+';else WANSECS=$((NOWSEC-WANSEC));WANPRE="";fi
    D=$((WANSECS/86400));if [ $D -ge 2 ];then WANTIME=${D}d;else
    H=$((WANSECS/3600));if [ $H -ge 2 ];then WANTIME=${H}h;else
    M=$((WANSECS/60));if [ $M -ge 2 ];then WANTIME=${M}m;else
    WANTIME=${WANSECS}s;fi;fi;fi
    #WANDATE=$(date -r /var/lib/misc/wantime '+%y%m%d.%H:%M')
    
    if [ -r /tmp/script_fire.sh ];then
    FIRSEC=$(date -r /tmp/script_fire.sh '+%s')
    if [ $FIRSEC -lt 999999 ];then FIRSECS=$FIRSEC;FIRPRE='+';else FIRSECS=$((NOWSEC-FIRSEC));FIRPRE="";fi
    D=$((FIRSECS/86400));if [ $D -ge 2 ];then FIRTIME=${D}d;else
    H=$((FIRSECS/3600));if [ $H -ge 2 ];then FIRTIME=${H}h;else
    M=$((FIRSECS/60));if [ $M -ge 2 ];then FIRTIME=${M}m;else
    FIRTIME=${FIRSECS}s;fi;fi;fi
    else FIRTIME="?"
    fi
    
    while [ ${#CON} -lt 3 ];do CON=" $CON"; done
    OUTLINE="$DATESTAMP$USE $CON $F $LOAD $P $FIRPRE$FIRTIME $WANPRE$WANTIME $BOOTIME ${Z# }"
    if [ "$DIR" != "" ];then echo "VIT: $OUTLINE" > $DIR/$HHMM; exit; fi
    if [ "$LOGGER" != "" ];then logger -p info -t VIT "$OUTLINE"; exit; fi
    case "$(cat /proc/$PPID/cmdline)" in
    */tmp/sc[hr]*.sh) logger -p info -t VIT "$OUTLINE" ;; # Scheduler / Adm Script
    *)                echo             "VIT: $OUTLINE" ;; # Interactive Shell
    esac
    if [ "$SLEEPSEC" == "" ];then exit; else sleep $SLEEPSEC; fi
    done
    
    Any feedback is much appreciated, including any findings that may be discovered, using this monitoring tool.
     
  2. rhester72

    rhester72 Network Guru Member

    Where is script_fire.sh and what does it do? :) vit seems to have a dependency on it.

    Rodney
     
  3. Planiwa

    Planiwa LI Guru Member

    Ah -- Thank you very much. Fixed now.

    This file is in /tmp and it gets touched when the FW script is (re)run, thus providing a clue to when the Firewall went up, most recently.

    Do you not have a Firewall script? Or does your version maybe put the corresponding file elsewhere?

    Might it be in /var/lib/misc?
     
  4. Toastman

    Toastman Super Moderator Staff Member Member

  5. Planiwa

    Planiwa LI Guru Member

    Here is the latest . . .

    Here is a new posting, rather than yet another edit:

    Code:
    # vit: Vital Statistics # Planiwa 2009-12-01
    
    INTERFACES="ppp0 eth1"; HALF=500; DIVI=1000; CONDIV=1; COUNTWHAT="bytes" # Defaults
    while [ $# -gt 0 ]
    do
    	case "$1" in
    	-*elp|-\?|-h) echo "Syntax		vit [-h] [-H] [-k] [-p] [-t] [-dir] [-I] [Interface ...]
    -h		This help
    -H		Don't show title headers
    -k		Show kiloBytes instead of MegaBytes, or kiloPackets instead of Packets
    -c		Show connections/100 instead of connections
    -p		Packets instead of Bytes
    -d		Prepend Datestamp
    -s SLEEPSEC	Repeat every SLEEPSEC seconds, until interrupted
    -l              Send output to syslog
    -dir DIR	Write output to file HHMM in directory DIR.
    		NB: Use CRU in INIT script to control exact Minute!  (Scheduler is unpredictable.)
    		eg: cru a Vit '01,16,31,46 * * * * /jffs/vit -d -p -dir /jffs/vitlog'
    -M		Show all Memory measures: Free Cached Buffered Active Inactive (in KiBi)
    -I		Show all Interfaces
    Interface ...	one or more interfaces.  Default: $INTERFACES
    " ;exit;;
        -H) NOHEADERS="Y";;
        -c) CONDIV=100;;
        -k) HALF=0; DIVI=1; KILO="kilo";; ### meaning of kilo is reversed for packets and bytes
        -p) COUNTWHAT="packets";;
        -M) ALLMEM="Y";;
        -I) INTERFACES="$(ifconfig|awk '/Link/{print$1}')";;
        -dir) DIR=$2; mkdir -p $DIR; HHMM=$(date '+%H%M'); shift;; ### write to file HHMM in $DIR
        -d) DATESTAMP=Y;;
        -l) LOGGER=Y;;
        -s) SLEEPSEC=$2;shift;;
        -s*) SLEEPSEC=${1#-s};;
    	-*) echo "vit: unrecognized paramter: $1"; set -- SHIFT -h;;
        *)  INTERFACES="$*"; break ;;
        esac
        shift
    done
    if [ "$NOHEADERS" == "" ];then
    case $ALLMEM in
    Y) echo "LANClients Connections Free,Cached,Buffered,Active,Inactive(KiBi) Load Processes FWUptime WANUptime SysUptime Interface:Tot-Up ...";;
    *) echo "LANClients Connections Free+Cached,Free(KiBi) Load Processes FWUptime WANUptime SysUptime Interface:Tot-Up ...";;
    esac; fi
    case $KILO$COUNTWHAT in
    kilopackets) HALF=500; DIVI=1000;;
    packets) HALF=0; DIVI=1;;
    esac
    
    while :;do
    if [ "$DATESTAMP" != "" ];then DATESTAMP="$(date '+%y%m%d_%H:%M:%S') "; fi
    
    set -- $(cat /proc/meminfo)
    case $ALLMEM in
    Y) M="F:${22} C:${31} B:${28} A:${37} I:${40}" ;;
    *) M="FC:$((${22}+${31})) F:${22}" ;;
    esac
    
    if [ -r /proc/sys/net/ipv4/ip_conntrack_count ]
    then CON=$(cat /proc/sys/net/ipv4/ip_conntrack_count) ### Thanks, Teddy Bear!
    else CON=$(cat /proc/net/ip_conntrack | wc -l)        ### This is very costly
    fi
    CON=$((CON/CONDIV))
    
    USE=$(arp|awk '/vlan1/{next};{N[$NF]++};END{for (i in N) print i ":" N[i]}')
    
    
    set -- $(cat /proc/loadavg)
    P=$4
    LOADL=${1%.*}; LOADL=${LOADL#0}; LOADR=${1#*.}; LOADR=${LOADR#0};
    LOADR=$(((LOADR+5)/10)); case $LOADR in 10)LOADR=9;;esac; LOAD=$LOADL.$LOADR
    
    X="";Y="";Z=""; for IF in $INTERFACES;do
      set -- $(ifconfig $IF |grep "$COUNTWHAT:")
      ### Beware: counters are reset at 4.3 GB (mod 2^32) ###
      case $# in
      0) Z="$Z $IF:Down";;
      *) case $COUNTWHAT in
         bytes)   RX=$2; TX=$6;;
         packets) RX=$2; TX=$8;;
         esac
         case $IF in ## these need some work, especially with unusual hardware or setups such as MLPPP
         ppp0|eth0|vlan1) X="$X $IF ${RX#*:} ${TX#*:}";; ## Up from router
         eth1|br0|*)      X="$X $IF ${TX#*:} ${RX#*:}";; ## Down from router
         esac;;
      esac
    done
    set -- $X; for X in $*;do
      case $X in
      [0-9]*) case $KILO$COUNTWHAT in
              *bytes)      if [ ${#X} -le 3 ];then X=0;else X=${X%???};fi; Y="$Y $(((X+$HALF)/$DIVI))";;
              *packets) Y="$Y $(((X+$HALF)/$DIVI))";;
              esac;;
      *)      Y="$Y $X";;
      esac
    done
    set -- $Y
    while [ $# -ge 3 ];do
      Z="$Z $1:$(($2+$3))-$3"
      shift 3
    done
    
    NOWSEC=$(date '+%s')
    
    BOOSEC=$(cat /proc/stat|grep btime); BOOSEC=${BOOSEC#* }
    BOOSECS=$((NOWSEC-BOOSEC))
    D=$((BOOSECS/86400));if [ $D -ge 2 ];then BOOTIME=${D}d;else
    H=$((BOOSECS/3600));if [ $H -ge 2 ];then BOOTIME=${H}h;else
    M=$((BOOSECS/60));if [ $M -ge 2 ];then BOOTIME=${M}m;else
    BOOTIME=${BOOSECS}s;fi;fi;fi
    #BOODATE=$(cat /proc/stat|grep btime|awk '{print strftime("%m%d.%H:%M",$2)}')
    
    WANSEC=$(date -r /var/lib/misc/wantime '+%s')
    if [ $WANSEC -lt 999999 ];then WANSECS=$WANSEC;WANPRE='+';else WANSECS=$((NOWSEC-WANSEC));WANPRE="";fi
    D=$((WANSECS/86400));if [ $D -ge 2 ];then WANTIME=${D}d;else
    H=$((WANSECS/3600));if [ $H -ge 2 ];then WANTIME=${H}h;else
    M=$((WANSECS/60));if [ $M -ge 2 ];then WANTIME=${M}m;else
    WANTIME=${WANSECS}s;fi;fi;fi
    #WANDATE=$(date -r /var/lib/misc/wantime '+%y%m%d.%H:%M')
    
    if [ -r /tmp/script_fire.sh ];then
    FIRSEC=$(date -r /tmp/script_fire.sh '+%s')
    if [ $FIRSEC -lt 999999 ];then FIRSECS=$FIRSEC;FIRPRE='+';else FIRSECS=$((NOWSEC-FIRSEC));FIRPRE="";fi
    D=$((FIRSECS/86400));if [ $D -ge 2 ];then FIRTIME=${D}d;else
    H=$((FIRSECS/3600));if [ $H -ge 2 ];then FIRTIME=${H}h;else
    M=$((FIRSECS/60));if [ $M -ge 2 ];then FIRTIME=${M}m;else
    FIRTIME=${FIRSECS}s;fi;fi;fi
    else FIRTIME="?"
    fi
    
    while [ ${#CON} -lt 3 ];do CON=" $CON"; done
    OUTLINE="$DATESTAMP$USE $CON $M $LOAD $P $FIRPRE$FIRTIME $WANPRE$WANTIME $BOOTIME ${Z# }"
    if [ "$DIR" != "" ];then echo "VIT: $OUTLINE" > $DIR/$HHMM; exit; fi
    if [ "$LOGGER" != "" ];then logger -p info -t VIT "$OUTLINE"; exit; fi
    case "$(cat /proc/$PPID/cmdline)" in
    */tmp/sc[hr]*.sh) logger -p info -t VIT "$OUTLINE" ;; # Scheduler / Adm Script
    *)                echo             "VIT: $OUTLINE" ;; # Interactive Shell
    esac
    if [ "$SLEEPSEC" == "" ];then exit; else sleep $SLEEPSEC; fi
    done
    
     
  6. jan.n

    jan.n Addicted to LI Member

    I just realized I had started it manually and that it's no longer running since the last reboot. What do I have to configure so this runs after reboots?
     
  7. Planiwa

    Planiwa LI Guru Member

    You could put something like ...

    Place the command in /jffs/vit
    (make sure it's executable)

    Then invoke it with the Scheduler.

    Or else invoke it with cron -- set this up in the Init Script like this:


    I would, however, strongly advise putting it in /root/vit first, making it executable, and getting familiar with it by invoking it from the command line with arguments that reflect your needs.

    For example:

    Code:
    [B]vit -h[/B]
    One might even look at the (comments in the) code.

    Vit consumes resources, so I would not run it constantly, unless the system was under intense scrutiny.
     
  8. trevorw

    trevorw LI Guru Member

    Excellent script Planiwa! Do you have any other to share?

    @jan.n
    Just add your script through the web interface to startup (Administration/Scripts. At init you can do a cru a "..." your vit script).
     
  9. trevorw

    trevorw LI Guru Member

    Resuscitating this thread - vit.sh doesn't seem to be working on the Tomato K26 - probably because of the kernel changes. Can anybody update the script please?

    Thanks,
     

Share This Page