1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

how to monitor traffic from each mac address?

Discussion in 'Tomato Firmware' started by slambert, Dec 4, 2008.

  1. slambert

    slambert Guest

    IN the tomato wiki under bandwidth monitoring it says:

    But it doesn't explain.

    I've googled for this and searched these forums and haven't been able to figure out how I can monitor the bandwidth use for each mac address.

    Thanks!
     
  2. HawkMan79

    HawkMan79 Addicted to LI Member

    I've also been curious about this and unable to figure it out.

    any help would be appreciated
     
  3. HawkMan79

    HawkMan79 Addicted to LI Member

    Well actually according to the manual this isn't mentiond in relation to QoS, but rather the bandwidth monitor.
     
  4. madsul

    madsul LI Guru Member

    Yes I was just reading this and trying to figure out how it is done. Can anyone help?
     
  5. Planiwa

    Planiwa LI Guru Member

    Defining a QoS "Class" for a host just to watch relative ephemeral upstream-only traffic in a pie chart can be of some limited use. It can show how out-of-control BT users can use up large amounts of connections while not really moving much data.

    If one wants to know traffic over time, for each host, (i.e. MB per user per hour), one can use iptables and some programs to collect, archive, and display the data.

    Simply assign a static IP address to each host. Then you can keep track by IP suffix. For example, .110 - .119 for the first floor, .120 - .129 for the second floor, etc. The following iptables rules collect the data for a particular set of 10 users from 192.168.0.100 - 192.168.0.132:

    Code:
    for SD in s d; do
      iptables -N traffic_$SD
      iptables -I FORWARD -j traffic_$SD
      for IP in 100 103 105 108 120 122 124 126 130 132; do
        iptables -A traffic_$SD -$SD 192.168.0.$IP
      done
    done
    You can run this as a Firewall script.

    You can feed this to a simple awk script to collect the data periodically:

    Code:
    for SD in s d;do iptables -L    traffic_$SD -vnx;done
    (add a -Z after the -L to reset the counters after reading them)
     
  6. Eleseo

    Eleseo Addicted to LI Member

    Planiwa, Thanks for the answer and sorry for the basic question, but
    ¿after run as a Firewall script, how can I see the result?,
    ¿how can access to collect the data periodically?, ¿by telnet? ¿..?. Thanks.
     
  7. Planiwa

    Planiwa LI Guru Member

    I find ssh / scp a whole lot easier to use than telnet. Have you used the command line interface at all?

    I can show you how to collect the data into a file on the router, and how to get it off the router with scp, for prettyprinting, etc., if that's what you need.

    There are other ways of getting the data off the router, that may suit you better than ssh. Telnet by itself is not great for data transfer, and hazardous if you're not inside the local network. :)

    More tomorrow ...
     
  8. Planiwa

    Planiwa LI Guru Member

    Here's the script to collect the data every hour:

    Code:
    # Traffic Accounting to be run Hourly at HH:59 -- Planiwa 2009.01.06
    # Appends to file /var/traffic/CCYYMM a line: CCYYMMDD.HH [IP-suffix dst-MB src-MB] ...
    
    [ -d /var/traffic ] || mkdir /var/traffic; cd /var/traffic
    set -- $(date '+%Y%m %d.%H'); CCYYMM=$1; DDHH=$2
    sleep 59
    
    for SD in s d;do iptables -L -Z traffic_$SD -vnx;done | awk '
    BEGIN {MB=2^20;KB=2^10; PRECISION=MB; SUBNET="192.168."} # adjust?
    $8 ~ SUBNET {if ($1!=0) D[substr($8,11)]=$2; next}
    $7 ~ SUBNET {if ($1!=0) S[substr($7,11)]=$2; next}
    END {
      printf "%s%s ", "'$CCYYMM'", "'$DDHH'"
      for (IP in D)if((D[IP]>=PRECISION/2)||(S[IP]>=PRECISION/2))
        printf("%s %d %d ",IP,D[IP]/PRECISION+.5,S[IP]/PRECISION+.5);print "" # MB
    }' >>$CCYYMM
    
    Here's what it looks like:

    # tail -24 /var/traffic/200901
    20090111.00 103 139 2 105 93 65 108 6 1
    20090111.01 103 71 1 108 8 0
    20090111.02 100 1 2 124 1 0 108 11 1 126 3 0
    20090111.03 100 2 29 124 2 0
    20090111.04
    20090111.05 100 2 26
    20090111.06 100 0 2
    20090111.07 100 0 1
    20090111.08 100 1 0
    20090111.09 103 28 1 105 68 2 126 5 1
    20090111.10 100 0 1 103 56 8 105 60 28 126 419 107
    20090111.11 100 82 4 103 1 0 105 227 59 124 8 1 108 19 3 126 439 7
    20090111.12 100 0 6 102 23 1 105 372 82 124 20 1 108 19 2 126 241 18
    20090111.13 102 51 2 103 30 10 105 123 76 124 3 1 108 131 25 126 122 6
    20090111.14 102 137 5 103 5 2 105 80 75 108 105 38 126 16 19
    20090111.15 100 0 3 102 118 5 105 35 74 124 167 4 108 110 9 126 17 10
    20090111.16 102 111 5 105 5 73 108 129 5 126 7 5
    20090111.17 102 220 10 105 2 6 108 16 3 126 3 5
    20090111.18 100 1 8 102 17 2 105 1 0 108 2 0 126 1 3
    20090111.19 100 0 3 105 61 42 108 1 0 126 1 2
    20090111.20 100 0 5 105 217 76 108 21 167 126 3 2
    20090111.21 100 0 2 102 67 2 120 6 1 105 179 73 108 35 43 126 20 3
    20090111.22 100 146 9 120 4 1 105 275 76 108 75 6 126 5 1
    20090111.23 100 165 20 102 206 6 105 272 55 124 1 0 108 23 2

    Here's how to get all the traffic files from the router into a directory on your computer called TRAFFIC/ --

    Code:
    scp -P2222 root@router:/var/traffic/*  TRAFFIC/
    To get the collector script to run at HH:59, you can use cru ...

    And to get it on the router, you might use scp to place it in /root or you might wget it from a web server that you control, or ...
     
  9. Eleseo

    Eleseo Addicted to LI Member

    Very interesting !!

    I had used the command line interface before, but basically.

    The only problem than I see, is the limit memory in the router.

    ¿Is posible that the script collect the data every hour and then "move" the traffic file from the router into a FTP directory or to CIFS client ( in Tomato case )?.

    Thanks.
     
  10. colinrocks

    colinrocks Addicted to LI Member

    Hi guys... This is almost exactly what I've just been experimenting with...

    <snip old, poorly hacked together, code>

    I'd love to knock up an awk script to convert it into something more readable, with calculated bandwidth percentages and historical data, but this does my needs nicely at the moment... hope it helps!

    EDIT: Well, it started bugging me, so I had a play with awk... and now I've got a much cleaner solution, with 60 mins worth of minute-by-minute logging and clean, tabular (rolling) output. See attached for my new script, and sample output. If anyone's interested, the values are bytes per second, and are in "downloaded|uploaded" layout... and there's meant to only be traffic to one computer, it's the only one thats online at the moment :)

    Just out of interest, is there any sort of standard formatting for handy Tomato scripts? Or repository/collection I can add it to? I've seen a few requests for bandwith-per-ip functionality, and this its the nail on the head!

    Cheers,
    Colin
     

    Attached Files:

  11. Eleseo

    Eleseo Addicted to LI Member

    Hello,

    I know I asking too much but, ¿is posible with a remote FTP server?

    Thanks again !!
     
  12. Planiwa

    Planiwa LI Guru Member

    Yes. Good idea. Should be able to do so with ftpput and ftpget.
     
  13. Eleseo

    Eleseo Addicted to LI Member

    Thank to Planiwa and Colin for these greats scripts !!

    I need some help for using "cru" to move the log files to a external FTP server. :confused:
     
  14. colinrocks

    colinrocks Addicted to LI Member

    Hey guys, I'm just testing an hourly version with FTP upload... and some more config options too - will post when it's working nicely :D
     
  15. Eleseo

    Eleseo Addicted to LI Member

    Thanks Colin !
     
  16. colinrocks

    colinrocks Addicted to LI Member

    Hi guys,

    Sorry for the slow response, but attached is the latest version - the script is in 2 parts, the top bit goes in the "Firewall" tab (I think) and the 2nd bit goes in the "Wan Up" section. I had problems keeping the firewall rules in place if the firewall was changed, but I think this fixes it.

    Don't forget to update the "setup" section at the top of each script!

    If you want to use the FTP upload when the stats are updated, simply fill in the FTP details, and uncomment the "ftpput" line (delete the # at the beggining of the line):

    Code:
    #ftpput -u "$FTP_USER" -p "$FTP_PASS" $FTP_HOST $FTP_DEST $BWLOG
    Let me know how you get on :smile:

    Cheers,
    Colin
     

    Attached Files:

  17. bandyta

    bandyta Addicted to LI Member

    Collins one question:

    first section:

    Code:
    # Setup:
    PREFIX=10.0.0			# first octets of IPs to monitor
    IPS="1 2 3 4 5 6 253 254"	# last octet of monitored IPs
    
    # Add chains and rules:
    # (Thanks to Planiwa for these)
    for SD in s d; do
      iptables -N traffic_$SD
      iptables -I FORWARD -j traffic_$SD
      for IP in $IPS; do
        iptables -A traffic_$SD -$SD $PREFIX.$IP
      done
    done
    ############

    second section:
    Code:
    # Setup:
    PREFIX=10.0.0			# first octets of IPs to monitor
    IPS="1 2 3 4 5 6 253 254"	# last octet of monitored IPs
    BWLOG=/var/wwwext/traffic.txt	# could be /cifs1/filename for saving to a CIFS mounted PC share
    CRON_HOUR=*			# which hours to run (* = all; 9-17 = 9am to 5pm; */2 = 2,4,6,8,10,12...)
    CRON_MIN=0			# which mins to run (* = all; 0 = 'oclock; 0,15,30,45 = every 15 mins)
    ENTRIES=24			# how many entries/lines to keep in the rolling file
    
    FTP_USER=username		# FTP details
    FTP_PASS=password		# 
    FTP_HOST=hostname		#
    FTP_DEST=folder			#
    
    # Create logfile header & lastrun file:
    mkdir /var/wwwext
    echo -ne "Time:\t" > $BWLOG
    for IP in $IPS; do echo -ne "$PREFIX.$IP:\t" >> $BWLOG; done
    echo "" >> $BWLOG
    date +%s > $BWLOG.run
    
    # Create trafic logging script:
    cat <<END_OF_SCRIPT >/tmp/home/root/bw_logger.sh
      ### Simple Bandwith Logging Script ###
      # keep header:
      head -1 $BWLOG > $BWLOG.tmp
    
      # calculate dwell time for bps
      NOW=\$(date +%s)
      export DWELL=\$(expr \$NOW - \$(cat $BWLOG.run))
      echo \$NOW > $BWLOG.run
    
      # pull out the iptables data, format with awk:
      # (modified from Plainwa's script)
      for SD in s d;do iptables -L -Z traffic_\$SD -vnx;done | awk '
      \$7 ~"$PREFIX" { I[\$7]++; S[\$7]=\$2/ENVIRON["DWELL"]; next }
      \$8 ~"$PREFIX" { I[\$8]++; D[\$8]=\$2/ENVIRON["DWELL"]; next }
      END { printf("%s\t",strftime("%H:%M"));for (IP in I) printf("%4d|%-4d\t",D[IP],S[IP]);print ""}
      ' >> $BWLOG.tmp
    
      # trim the file to the last n entries:
      tail -n +2 $BWLOG | head -$ENTRIES >> $BWLOG.tmp
      mv $BWLOG.tmp $BWLOG
    
      # FTP the file somewhere:
      #ftpput -u "$FTP_USER" -p "$FTP_PASS" $FTP_HOST $FTP_DEST $BWLOG
      ### End Of Script ###
    END_OF_SCRIPT
    
    # Schedule logging:
    cru a BWMonitor "$CRON_MIN $CRON_HOUR * * * sh /tmp/home/root/bw_logger.sh"
    
    yes or no ?
    thx
     
  18. colinrocks

    colinrocks Addicted to LI Member

    Yes - spot on :thumbup:
     
  19. bandyta

    bandyta Addicted to LI Member

    Damn :(
    dont work :(

    this i my settings -->
     

    Attached Files:

  20. colinrocks

    colinrocks Addicted to LI Member

    Hmm... do you get anything in the /var/wwwext/traffic.txt file? You'll need to either SSH or telnet to the router, and then:

    Code:
    cat /var/wwwext/traffic.txt
    to view it, or go to http://routerip/ext/traffic.txt (this might only work if you're using a modified firmware, not sure).

    What firmware are you using? Anything interesting in the logs?
     
  21. bandyta

    bandyta Addicted to LI Member

    I use Victed Mod.
    Telnet Daemon on Tomato is ON.

    Logs:
    Code:
    Jan 16 00:00:57 ? user.warn kernel: nvram_commit(): init
    Jan 16 00:00:58 ? user.warn kernel: nvram_commit(): end
    Jan 16 00:01:19 ? user.warn kernel: nvram_commit(): init
    Jan 16 00:01:21 ? user.warn kernel: nvram_commit(): end
    
     
  22. colinrocks

    colinrocks Addicted to LI Member

    Bandyta - is there anything in /var/wwwext/traffic.txt (or http://routersIP/ext/traffic.txt)? If that's got the header (1st line with tab seperated IPs in) then we're getting somewhere, at least the code has run... if there's any lines under it then the cron's working.

    If you're just getting rows of "0|0 0|0 0|0" then you'll need to check the IPs you've used in the setup sections... if you've got upload and download values (i.e. anything other than zeros) then it looks like the FTP didn't work... time to check your FTP details.

    If /var/wwwext/traffic.txt is empty (or missing), we might need to change the traffic.txt location... what router do you have?

    Cheers,
    Colin
     
  23. bandyta

    bandyta Addicted to LI Member

    I have Linksys WRT54GL

    Or to the run this script is needed to mount CIFS on my computer ?
     
  24. colinrocks

    colinrocks Addicted to LI Member

    No, should run fine without a CIFS mount... see if you can find (and post) the first few lines from /var/wwwext/traffic.txt or http://routersIP/ext/traffic.txt - that'll let us know if the script is running or not, and (hopefully) where it's failing.

    Cheers,
    Colin
     
  25. rizsher

    rizsher Network Guru Member

    I can't seem to get very far either. These are my scripts:

    Under Firewall:

    # Setup:
    PREFIX=192.168.1 # first octets of IPs to monitor
    IPS="100 110 120 121 126 127" # last octet of monitored IPs

    # Add chains and rules:
    # (Thanks to Planiwa for these)
    for SD in s d; do
    iptables -N traffic_$SD
    iptables -I FORWARD -j traffic_$SD
    for IP in $IPS; do
    iptables -A traffic_$SD -$SD $PREFIX.$IP
    done
    done

    and WAN Up:

    # Setup:
    PREFIX=192.168.1 # first octets of IPs to monitor
    IPS="100 110 120 121 126 127" # last octet of monitored IPs
    #BWLOG=/cifs1/BWDATA.XLS
    BWLOG=/var/wwwext/traffic.txt # could be /cifs1/filename for saving to a CIFS mounted PC share
    CRON_HOUR=* # which hours to run (* = all; 9-17 = 9am to 5pm; */2 =

    2,4,6,8,10,12...)
    CRON_MIN=0 # which mins to run (* = all; 0 = 'oclock; 0,15,30,45 = every 15

    mins)
    ENTRIES=24 # how many entries/lines to keep in the rolling file

    FTP_USER=username # FTP details
    FTP_PASS=password #
    FTP_HOST=hostname #
    FTP_DEST=folder #

    # Create logfile header & lastrun file:
    mkdir /var/wwwext
    echo -ne "Time:\t" > $BWLOG
    for IP in $IPS; do echo -ne "$PREFIX.$IP:\t" >> $BWLOG; done
    echo "" >> $BWLOG
    date +%s > $BWLOG.run

    # Create trafic logging script:
    cat <<END_OF_SCRIPT >/tmp/home/root/bw_logger.sh
    ### Simple Bandwith Logging Script ###
    # keep header:
    head -1 $BWLOG > $BWLOG.tmp

    # calculate dwell time for bps
    NOW=\$(date +%s)
    export DWELL=\$(expr \$NOW - \$(cat $BWLOG.run))
    echo \$NOW > $BWLOG.run

    # pull out the iptables data, format with awk:
    # (modified from Plainwa's script)
    for SD in s d;do iptables -L -Z traffic_\$SD -vnx;done | awk '
    \$7 ~"$PREFIX" { I[\$7]++; S[\$7]=\$2/ENVIRON["DWELL"]; next }
    \$8 ~"$PREFIX" { I[\$8]++; D[\$8]=\$2/ENVIRON["DWELL"]; next }
    END { printf("%s\t",strftime("%H:%M"));for (IP in I) printf("%4d|%-4d\t",D[IP],S[IP]);print ""}
    ' >> $BWLOG.tmp

    # trim the file to the last n entries:
    tail -n +2 $BWLOG | head -$ENTRIES >> $BWLOG.tmp
    mv $BWLOG.tmp $BWLOG

    # FTP the file somewhere:
    #ftpput -u "$FTP_USER" -p "$FTP_PASS" $FTP_HOST $FTP_DEST $BWLOG
    ### End Of Script ###
    END_OF_SCRIPT

    # Schedule logging:
    cru a BWMonitor "$CRON_MIN $CRON_HOUR * * * sh /tmp/home/root/bw_logger.sh"

    Tried both : #BWLOG=/cifs1/BWDATA.XLS and BWLOG=/var/wwwext/traffic.txt. No file created on CIFS1 (where I actually save the BW Monitoring data w/o a problem), nor is there a wwwext folder.

    This is on a WRT54GS with: Tomato Firmware v1.23.8025, Addons by Victek@gmail.com

    Any ideas?
     

Share This Page