1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

RRD Tool Quality Graph - Would this be possible in Tomato?

Discussion in 'Tomato Firmware' started by Mysteron, Feb 2, 2012.

  1. Mysteron

    Mysteron Networkin' Nut Member

    As the title suggests, I'm wondering if it's possible to create something like the pfsense line quality graph in tomato. I gather pfsense uses RRD tool and I can see there's an optware package (in fact two, RRD Tool and RRDCollector) available, but I have no idea where to go from there.

    Any help would be appreciated.

    Thanks

    Asus WL-500gp v2 with Shibby Mod
     
  2. Planiwa

    Planiwa LI Guru Member

    You mean like the DMT charts in my avatar? Those were done entirely on the router using the shell.
     
  3. Mysteron

    Mysteron Networkin' Nut Member

    I'm not really sure what those charts are showing? The kind of thing I'm after is something similar to the image. As far as I know, it just 'pings' various points and sends the results to a database, the report is based off this data.

    [​IMG]
     
  4. Planiwa

    Planiwa LI Guru Member

    FWIW, I have something called visiping which displays a progress meter and then charts the summaries on a logarithmic scale. I find that Median and 90th Percentile are far more informative than Mean, and that the latency can range from 8ms to 10s or more. By default it pings the WAN gateway.

    visiping0204.png
     
  5. rhester72

    rhester72 Network Guru Member

    visiping looks very nice. Source? :)

    Rodney
     
  6. Mysteron

    Mysteron Networkin' Nut Member

    That does look very interesting, would you mind sharing the method to replicate.

    Thanks
     
  7. Planiwa

    Planiwa LI Guru Member

    I'm in the middle of reworking it, so I had to stabilize some bits. I've incorporated some other statistics: Number of (wifi) users, number of (NAT) connections, WAN traffic. It needs to call an external script, wantraf.

    Here is what it looks like now. At the bottom you can see the "progress meter".

    visiping.png

    Here is wantraf:
    Code:
    WANIF=$(nvram get wan_iface)
    S=${1-20}
    D=$((1000*$S/8))
    set -- $(ifconfig $WANIF)
    RX=${30#*:}
    TX=${34#*:}
    sleep $S
    set -- $(ifconfig $WANIF)
    RX1=${30#*:}
    TX1=${34#*:}
    echo  "$(((TX1-TX)/$D)) $(((RX1-RX)/$D)) kb/s"
    
    And here is visiping:
    Code:
    : # visiping # Planiwa 2012.02.08 (a work in progress)    (needs to call wantraf)
    
    INTERVAL=1; COUNT=3600; ALARM=0; DETAIL=0
    while [ $# -gt 0 ]
    do
        case "$1" in
    
        -*elp|-\?|-h) echo "
    Visiping shows each ping and charts a summary line every 60 pings, showing Min, Median, Mean, 90%, Max.
    Uses ANSI SGR for colour and placement.
    Syntax visiping [-i INTERVAL (1)] [-c COUNT (3600)] [-d DETAIL (0)] [-A] [HOST (Gateway)]
    -h          This help
    -i INTERVAL Wait INTERVAL seconds between pings.  Default=1.  (Tomato Busybox ping lacks this functionality.)
    -c COUNT    Send this many pings.  Default=3600.
    -A          Sound an alarm whenvever there is no response for whatever reason.
    -d DETAIL   -1: no detail; 0: default; 1: each ping time; 2: unreachable; 3: timeouts; 4: all ping messages.
    HOST        Default=Gateway (For Tomato only.)
    " ;exit;;
    
        -A)   ALARM=1;;
        -d)   DETAIL=$2;shift;;
        -d?*) DETAIL=${1#-d};;
        -c)   COUNT=$2;shift;;
        -c?*) COUNT=${1#-c};;
        -i)   INTERVAL=$2;shift;;
        -i?*) INTERVAL=${1#-i};;
        -*)   echo "visiping: unrecognized parameter: $1"; set -- SHIFT -h;;
        *)    HOST="$1"; break ;;
        esac
        shift
    done
    case $DETAIL in 0) DETAIL="";; esac
    
    case "$INTERVAL" in
    1) ;;
    *) case "$(ping -h 2>&1)" in
       *-i*) INTER="-i$INTERVAL" ;;
       *) echo "The Busybox ping on this system does not understand '-i'. Later we will work around that."
          echo "Using 1s for now"; INTERVAL=1
             ;;
       esac
    esac
    
    PHOST="$HOST"
    case "$HOST" in
    "") HOST=$(nvram get wan_gateway_get 2>&1)
        case $? in
        0) PHOST="Gateway";;
        *) echo "Cannot determine Gateway.  Using 4.2.2.2"; echo; HOST=4.2.2.2;;
        esac
        ;;
    esac
    
    ### Kludge to cope with Busybox
    ESC="$(echo e | tr 'e' '\033')"
    CSI="$ESC["
    BRI="$ESC[1m"
    INV="$ESC[7m"
    RES="$ESC[m"
    CLI="$ESC[1G$ESC[K"
    CHA="$ESC[1G"
    RED="$ESC[1;31m"
    
    SETS=$(((COUNT+59)/60))
    ### case $(uname) in Linux) GAP="";; *) GAP="-i$PINGGAP";; esac # busybox ping is broken
    HHMM=$(date '+%H%M'); FILE=/tmp/visiping-$HHMM
    LASTIP=""
    
    SETSLEFT=$SETS
    while [ $SETSLEFT -gt 0 ];do
      ### case $SETSLEFT in $SETS) PREVHOST=$HOST;; esac
      ### if [ "$HOST" != "$PREVHOST" ];then echo "HOST: $PREVHOST -> $HOST";PREVHOST=$HOST;fi
    
          SETSLEFT=$((SETSLEFT-1))
    
     set -- $(ping -c1 -s28 $HOST 2>&1)# is this host reachable? What IP address?
     case $1 in
     ping:)echo "                                     $(date '+%m%d_%H:%M') $@"; sleep 60;continue;; ### FAILED
     esac
     IP=${10%:}
     case "$IP" in
     *.*.*.*);;
     *) IP="${3%:}" ;IP="${IP#(}";IP="${IP%)}" ; echo -n "${CSI}1A$RED!$RES${CSI}1B" ;;
     esac
     if [ "$IP" != "$LASTIP" ];then
             echo;echo "DateTime Usrs Cons  U kb/s D Min  50%  Mea  90%   Max Lost $INV$BRI $PHOST $RES $INV$BRI$IP$RES"
        LASTIP=$IP
     fi
    
          CONNECTIONS=$(cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count)
          WIFIUSERS=$(wl assoclist|wc -l)
    
          echo "$WIFIUSERS $CONNECTIONS $(sleep 15;wantraf 30)" > /tmp/visiping-vitals &
    
     case $DETAIL in *) MM=$(date '+%M'); FILE=/tmp/visiping-$MM ;; esac # For forensics within 1 hour.
     TIMES=0; echo    "">$FILE
     ping $INTER -c60 -s28 $HOST 2>&1 | while read LINE
          do
                set -- $LINE
    case $TIMES in
                0|20|40) case $DETAIL in ?)echo;;esac
            case $DETAIL in ""|?)
                         echo -n "$CHA$RED$((1+TIMES/20))$RES " >>/dev/tty;;
                         esac ;;
                esac
    case $3 in
    from) TIMES=$((TIMES+1));
         case $DETAIL in ""|?)
                           MS=${7#time=}; MS=${MS%.*}; echo -n "$MS ">>/dev/tty ;;
                      esac ;;
    Network|resolve|No) case $DETAIL in 1|2)echo; echo $*;echo;;esac
                                    case $ALARM in 1)echo -e -n "\a">>/dev/tty;; esac
           TIMES=$((TIMES+1)); echo -n "$INV$RED!$RES ">>/dev/tty ;;
    for)                case $DETAIL in 1|2|3)echo; echo $*;echo;;esac ;;
    "("*|ping|transmitted|=) case $DETAIL in 1|2|3|4)echo; echo $*;echo;;esac ;;
    *)                  case $DETAIL in ?)echo; echo $*;echo;;esac ;;
    esac
    echo    "$*" >>$FILE
     done
    
    # ping: sendto: Network is unreachable
    # ping: cannot resolve torix.net: Unknown host
    # ping: sendto: No route to host
    # PING torix.net (69.165.158.204): 56 data bytes
    # Request timeout for icmp_seq 6
    # 64 bytes from 69.165.158.204: icmp_seq=6 ttl=48 time=1702.440 ms
    # --- torix.net ping statistics ---
    # 34 packets transmitted, 19 packets received, 44.1% packet loss
    # round-trip min/avg/max/stddev = 100.183/525.669/2201.817/648.560 ms
    
    sleep $INTERVAL
       
    # continue
    
    sed -e 's/^.*time=/Z:\ /' -e 's/ ms$//' -e 's/^r/R/' $FILE | grep '[:%/]' | sort -k2 -n \
    | awk '
    BEGIN { MIN=0; MAX=0; MEDIAN=0; HOST="'$HOST'"; P90=0
            RED="\33[1;31m"; GRE="\33[1;32m"; YEL="\33[1;33m"; BLU="\33[1;34m"
    MAG="\33[1;35m"; CYA="\33[1;36m"; INV="\33[7m"; RES="\33[m"; CLI="\33[1G\33[K" }
    
    $1=="Z:" { MS=$2; RECD++; TOTMS+=MS   ### NR: 3 .. 62
    if (MIN==0) MIN=MS
    if (MS<MIN) MIN=MS
    if (MS>MAX) MAX=MS
    if (MEDIAN==0) if (RECD>=MEDIWHICH) MEDIAN=MS
    if (P90==0)    if (RECD>=P90WHICH) P90=MS
    next
    }
    $2=="packets" { SENT=$1; LOST=$1-$4; MEDIWHICH=int((SENT+1)/2); P90WHICH=int((SENT+1)/10*9) ; next} ### NR==1 !
    $1=="Round-trip" { MAMS=$4 ; next}          ### NR==2 !
    $1=="PING"{IP=$3; next}
    { ### errors
    ERRORS[$0]++
    }
    # { printf "%s %s %s %s %s      %s     ", CLI, MIN, MS, MAX, RECD, $0 }
    
    END {
    for (E in ERRORS)
    {print "";print INV " " ERRORS[E] " alerts: " IP RES, E}
    split(MAMS,AMAMS,"/")
    MINI=AMAMS[1]
    MEAN=AMAMS[2]
    MAXI=AMAMS[3]
    
    "date \"+%m%d_%H%M\"" | getline DATE
    printf ("%s%s", CLI, DATE)
    if (RECD==0) { print " " INV " Host Unreachable " RES; exit }
    MIN=int(.5+MIN)
    MED=int(.5+MEDIAN)
    MEA=int(.5+TOTMS/RECD)
    P90=int(.5+P90)
    MAX=int(.5+MAX)
    if (P90==0) P90=MAX
    HCMIN=humancount(MIN)
    HCMED=humancount(MED)
    HCMEA=humancount(MEA)
    HCP90=humancount(P90)
    HCMAX=humancount(MAX)
    
    SHOW=rep(HCMIN-1,".")                    GRE "<" RES ; LASTMARK=HCMIN
    if (HCMED>LASTMARK)           {SHOW=SHOW GRE rep(HCMED-LASTMARK-1,"<") RES CYA "|" RES; LASTMARK=HCMED}
    if (HCMEA<=HCP90)
     { if (HCMEA>LASTMARK)       {SHOW=SHOW CYA rep(HCMEA-LASTMARK-1,".") BLU "X" RES; LASTMARK=HCMEA}
       if (HCP90>LASTMARK)       {SHOW=SHOW MAG rep(HCP90-LASTMARK-1,">") MAG ">" RES; LASTMARK=HCP90} }
      else { if (HCP90>LASTMARK) {SHOW=SHOW MAG rep(HCP90-LASTMARK-1,">") MAG ">" RES; LASTMARK=HCP90}
     if (HCMEA>LASTMARK) {SHOW=SHOW CYA rep(HCMEA-LASTMARK-1,".") BLU "X" RES; LASTMARK=HCMEA} }
    if (HCMAX>LASTMARK)           {SHOW=SHOW RED rep(HCMAX-LASTMARK-1,".") RED "+" RES; LASTMARK=HCMAX}
    
    if (MIN==999) MIN=""
    if (MED==0) MED=""
    if (MEA==0) MEA=""
    if (P90==0) P90=""
    if (MAX==0) MAX=""
    
        "cat /tmp/visiping-vitals" |getline
    printf ("%3s%5s%5s%6s",$1,$2,$3,$4)
    
    printf ("%14s%15s%15s%15s%16s ", GRE MIN RES, CYA MED RES, BLU MEA RES, MAG P90 RES, RED MAX RES)
    if (LOST>0) printf("%s%2d%s ", INV, LOST, RES)
    else printf "   "
    print SHOW
    }
    
    function humancount(N) {
    # DB, Decibel if BASE=10, decioct if base=8, decibin for 2.  Let us use deciquint.
    BASE=5
    DB=int(.5+log(N)/log(BASE)*10)
    return DB
    }
    
    function rep(n,s, t) {
      while (n-- > 0)
        t = t s
        return t
      }
    '
    
    done ### SETSLEFT
    echo
    
    exit
    
    time=*) while [ ${E#*se} != q=$TIMES ] # seq=... and icmp_seq=...
    do
    TIMES=$((TIMES+1)); echo -n "$INV$RED.$RES ">>/dev/tty
                            echo -ne "\a">>/dev/tty; break 2
               case $TIMES in 0|20|40) case $DETAIL in 1)echo;;esac; echo -n "$CHA$RED$((1+TIMES/20))$RES " >>/dev/tty;;esac
    done
    TIMES=$((TIMES+1)); MS=${G#time=}; MS=${MS%.*}; echo -n "$MS ">>/dev/tty ;;
    esac
    echo    "$A $B $C $D $E $F $G $H">>$FILE
    done
    
    UPDATED 2010.02.09 10:39 EST
     
  8. rhester72

    rhester72 Network Guru Member

    Why not replace the file in /tmp with something like /tmp/visiping-vitals.$$ to avoid the concurrency issue (and delete it at exit)?

    Rodney
     
  9. Planiwa

    Planiwa LI Guru Member

    Yes, good idea.

    Actually, I had exactly that originally. Then I made some changes to reduce some risks, and to cope with the Busybox shell, etc. There are several fragments of code that I have removed, commented out, or changed for various reasons.

    The other file (which was originally named /tmp/visiping-$$) is now -MM which makes it possible to monitor and explore better, but can no longer handle concurrent processes. (Could at least have a lock to prevent 2nd process from starting.) Part of the reason for -MM is to allow a look at the actual ping lines, during and *after*. At least -MM makes at most 60 files and is unlikely to use too much space.

    I will almost certainly change this later. Others may want to do it sooner. :)
     
  10. Planiwa

    Planiwa LI Guru Member

    To generalize for WAN Interface -- you can replace the references to ppp0 in visiping with wantraf.
    wantraf:
    Code:
    WANIF=$(nvram get wan_iface)
    S=${1-20}
    D=$((1000*$S/8))
    set -- $(ifconfig $WANIF)
    RX=${30#*:}
    TX=${34#*:}
    sleep $S
    set -- $(ifconfig $WANIF)
    RX1=${30#*:}
    TX1=${34#*:}
    echo  "$(((TX1-TX)/$D)) $(((RX1-RX)/$D)) kb/s"
    
    This *may* work (unless ifconfig output differs, depending on interface, which is possible, and which would require more complex parsing).

    Perhaps someone with an interface other than ppp0 can report? [Programming note -- I'm aware that I am parsing the variables positionally, although they are clearly keyworded. I'm interested in a simple shell script keyword parsing method that works with the Busybox shell.]

    Note: I have updated the visiping source in place.
     
  11. Mysteron

    Mysteron Networkin' Nut Member

    Many thanks for posting this Planiwa, it's something I'd be most interested in getting up and running. My problem is, I'm not sure where to place this script, or how to invoke it? If I can get it going, I'll gladly report back. My interface type is pptp.
     
  12. Planiwa

    Planiwa LI Guru Member

    The more basic question is whether one has an ANSI-capable terminal (via ssh).
    Tools > System is not enough for visiping, because it does not support ANSI SGR.

    The obvious place for shell commands is in the home directory, /root, and then one can log in to the erouter via ssh, or run the command on a remote host via ssh.

    On a Mac, that would be via Terminal. On a MS system, it might be something like Putty -- but I don't know anything about MS.

    As for wantraf -- if I thought there was a need or interest, I could make it more robust, but thus far it seems that no one except perhaps Rodney has any use for it.

    This is a more robust wantraf:
    Code:
    WANIF=$(nvram get wan_iface)
    S=${1-20}
    D=$((1000*$S/8))
    set -- $(ifconfig $WANIF|grep 'RX bytes:')
    RX=${2#*:}
    TX=${6#*:}
    sleep $S
    set -- $(ifconfig $WANIF|grep 'RX bytes:')
    RX1=${2#*:}
    TX1=${6#*:}
    echo  "$(((TX1-TX)/$D)) $(((RX1-RX)/$D)) kb/s"
    
     

Share This Page