1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Per IP Bandwidth monitoring how to?

Discussion in 'Tomato Firmware' started by jdub8, Oct 1, 2011.

  1. jdub8

    jdub8 Addicted to LI Member

    I'm running tomato 1.28 and im pretty sure this is possible with scripting after you change some setting, i forget which.

    Can anyone outline how to set up my bandwidth graphs to have per-ip or per-mac address bandwidth graphs?
     
  2. shibby20

    shibby20 Network Guru Member

  3. eahm

    eahm LI Guru Member

    That looks amazing.
     
  4. shibby20

    shibby20 Network Guru Member

    indeed :)
     
  5. Toastman

    Toastman Super Moderator Staff Member Member

    You can try a test build from the website below. Reports so far say it is pretty much OK. (A few posts that the ipv6 firewall on this build is missing some necessary lines, so IPV6 isn't fully working apparently). Look at the readme file for the current state of development.
     
  6. jdub8

    jdub8 Addicted to LI Member

    Thank you this looks much more robust than the method I saw years ago that relied on scripting (uhg) and I never could get a good explination of how it worked or how to get it running.

    Which file should I download and which version should I use? I'm using a WRT54GL V1.1.

    I know what VPN means but SDHC and SNMP excape me.

    Do I need to do a NVRAM clear? Can I backup my configuration from tomato 1.28 and import it after flashing?
     
  7. Toastman

    Toastman Super Moderator Staff Member Member

    K2.4 MIPSR1 versions, 1.28.7628 std
     
  8. jdub8

    jdub8 Addicted to LI Member

    I don't see that version on the downloads link... I just installed the tomato-ND-1.28.0007-Std from the Tomato-Teaman-ND-1.28.0007.7z file. It's working great btw. Thanks a million!

    I know feature requests must annoy the hell out of you guys but something ive always wanted was for the bandwidth data to cache the last few minutes and be pre-loaded when you view it. IE the charts already have at least some data pre-loaded when you fire them up. 99% of the time I log into my router its to check my bandwidth usage, now that I can see per-ip stats (THANK YOU!) the only thing that would make it better is if I could see the last few minutes rather than the next few minutes going forward. That way I could tell which IP had been hogging the bandwidth and causing me to lag.
     
  9. teaman

    teaman LI Guru Member

    SNMP = http://en.wikipedia.org/wiki/Simple_Network_Management_Protocol (on Tomato-Teaman-ND builds, this feature/code comes from the tomato-shibby branch on GIT)
    SDHC = http://gemini.net.pl/~slodki/tomato-sdhc.html (requires hardware mod)
    I'd suggest taking a look at the IP Traffic/Last 24 Hours page, below the bandwitdh graphic, to the right, you can select how many hours/how much time you wish to see.

    Glad to hear you're enjoying that build! I should be posting a new version/build soon (Tomato-Teaman-ND-1.28.0008), which includes a few fixes that didn't make into v0007 as well as cstats V2.

    Major highlights/improvements on cstats V2:
    - internals/stats moved away from static arrays to an AVL binary tree (there used to be a hard-coded max of 64 IPs cstats would be able to track)
    - load/save stats is a whole more resilient to temporary failures (mostly, at startup, when external/remote filesystems are being mounted, etc...)
    - added 'auto-discover IPs' to IPTraffic monitor (previously, you had to explicitly enable tracking for each IP address you wanted to keep/track BW/IP traffic history)
    - preliminary testing indicates keeping track/stats for one IP address 'costs' 7920 bytes (memory is now allocated and/or freed dynamically as required)

    Have fun!
     
  10. mikester

    mikester Network Guru Member

    Enable logging of all http requests including blocked/dropped and it will be perfect!
     
  11. Thurman

    Thurman Networkin' Nut Member

    I'm running ND-MIPS32R1 K2.4 1.28.7628 Std and when it works, it's great. However, IP Traffic disappears and won't show anything after a few hours. I'm not sure what the trigger is, but then it will start re-appearing again.

    I'm not sure if this is the right place to let you know of issues, and I see you have an even newer build (407?) out, but I can't run it on my WRT54GL.

    I'm very interested in this functionality and would be willing to test it out. I especially am looking forward to the new functionality so I don't have to specify every IP staticly off the device list to get them to show up on the IP traffic reports.
     
  12. STEVEWEST

    STEVEWEST Networkin' Nut Member

    I am getting the same thing. After a few hours the graphs disappear. Is there any special setting or limits on IP Traffic monitoring. IE number of ip address?
     
  13. teaman

    teaman LI Guru Member

    Are you using cstats V1 or V2?
    When this happens, do you see any data on being shown/updated on IP Traffic->Real-time (ipt-realtime.asp)?
    Or perhaps on IP Traffic->Last 24h (ipt-24.asp)?
    Or even on the IP Traffic->Monthly history (ipt-monthly.asp)?
    Do you notice anything else that might seem 'odd'? Any weird/strange messages appearing (recently) on the system logs?

    Also, would you be able to check if you have an iptables rule similar to this one when the problem occurs?
    Code:
    root@none:/tmp/home/root# iptables -nvx -L | grep -i account
      79863 26338524            all  --  *      *      0.0.0.0/0            0.0.0.0/0          account: network/netmask: 192.168.1.0/255.255.255.0 name: lan
    root@none:/tmp/home/root#
    
    Or... if your router is indeed keeping IP traffic stats so cstats can read them?
    Code:
    root@none:/tmp/home/root# cd /proc/net/ipt_account
    root@none:/proc/net/ipt_account# ls -l
    -rw-r--r--    1 root    root            0 Oct 16 03:25 lan
    root@none:/proc/net/ipt_account# cat lan | wc -l
    256
    root@none:/proc/net/ipt_account#
    
    Or even... do you have a cstats daemon running when that happens?
    Code:
    root@none:/proc/net/ipt_account# ps | grep stats
      97 root      816 S    rstats
      115 root      1384 S    cstats
    1003 root      1172 S    grep stats
    root@none:/proc/net/ipt_account#
    
    Please let us know, so we can try reproduce the problem and/or learn a bit more about it (to eventually, fix that) ;-)
    There are possibly... three things that could be considered 'limitations':
    a) as posted before, preliminary testing indicates keeping track/stats for one IP address 'costs' 7920 bytes when cstats runs on my WRT54GL (K24 build)
    b) cstats V1 had a hard limit/max of 64 addresses it would be able to keep track - when those slots were all 'taken', it wouldn't begin/start keeping track of any other/new addresses, but would continue to keep track and update stats for the ones already 'known' - cstats V2 doesn't have such limitation (memory is allocated dynamically, as required, for each new IP it should be keeping history)
    c) please also keep in mind total/free/available memory/RAM should also be considered when talking about any 'limits' (specially when/if we're talking about running lots of processes/daemons, keeping track of lots of connections, etc... on devices with just 16M or RAM... such as mine).

    Cheers!
     
  14. STEVEWEST

    STEVEWEST Networkin' Nut Member

    I am using cstat V2 on Tomato v1.28.0905 MIPSR2-079V K26 USB AIO. I can reset the router and watch the graphs and everything is fine. When I switch menus the cstat data goes away, some menus say cstat is busy. Try refresh and nothing happens. Here is some data I collected.

    reset router All is good




    Model

    Asus RT-N16


    Chipset

    Broadcom BCM4716 chip rev 1 pkg 10


    CPU Freq

    480MHz


    Flash RAM Size

    32MB



    Time

    Sun, 16 Oct 2011 16:31:54 -0700


    Uptime

    0 days, 00:20:59


    CPU Load (1 / 5 / 15 mins)

    0.01 / 0.03 / 0.07


    Total / Free Memory

    124.11 MB / 106.79 MB(86.04%)



    Tomato v1.28.0905 MIPSR2-079V K26 USB AIO

    root@Tomato:/tmp/home/root# iptables -nvx -L | grep -i account

    3249 1262086 all -- * * 0.0.0.0/0 0.0.0.0/0 account: network/netmask: 192.168.1.0/255.255.255.0 name: lan

    root@Tomato:/tmp/home/root#

    root@Tomato:/tmp/home/root# cd /proc/net/ipt_account

    root@Tomato:/proc/net/ipt_account# ls -l

    -rw------- 1 root root 0 Oct 16 16:25 lan

    root@Tomato:/proc/net/ipt_account# ps | grep stats

    624 root 1028 S rstats

    1585 root 1048 S cstats --new

    2004 root 1716 S grep stats

    root@Tomato:/proc/net/ipt_account#

    After losing graphs and stats

    iptables -nvx -L | grep -i account

    41060 10588605 all -- * * 0.0.0.0/0 0.0.0.0/0 account: network/netmask: 192.168.1.0/255.255.255.0 name: lan

    ls -l

    -rw------- 1 root root 0 Oct 16 17:10 lan

    624 root 1028 S rstats

    1585 root 1072 S cstats --new

    2778 root 1716 S grep stats




    CPU Load (1 / 5 / 15 mins

    0.00 / 0.00 / 0.00





    Total / Free Memory

    124.11 MB / 106.48 MB(85.80%)



    Oct 16 18:00:01 Tomato cron.info crond[621]: crond: USER root pid 2673 cmd logger -p syslog.info -- -- MARK --

    Oct 16 18:00:01 Tomato cron.info crond[621]: crond: USER root pid 2675 cmd rcheck --cron

    Oct 16 18:03:01 Tomato daemon.warn miniupnpd[1318]: HTTP Connection from 192.168.1.35 closed unexpectedly

    Oct 16 18:03:01 Tomato daemon.warn miniupnpd[1318]: HTTP Connection from 192.168.1.35 closed unexpectedly

    Oct 16 18:15:01 Tomato cron.info crond[621]: crond: USER root pid 3177 cmd rcheck --cron

    Oct 16 18:30:01 Tomato cron.info crond[621]: crond: USER root pid 3693 cmd rcheck --cron

    Oct 16 18:45:01 Tomato cron.info crond[621]: crond: USER root pid 4208 cmd ntpsync --cron

    Oct 16 18:45:01 Tomato cron.info crond[621]: crond: USER root pid 4210 cmd rcheck --cron

    Oct 16 18:45:01 Tomato user.info ntpc[4209]: Time: Sun, 16 Oct 2011 18:45:01 -0700, no change was needed.

    Oct 16 19:00:01 Tomato cron.info crond[621]: crond: USER root pid 4737 cmd logger -p syslog.info -- -- MARK --

    Oct 16 19:00:01 Tomato cron.info crond[621]: crond: USER root pid 4739 cmd rcheck --cron

    Oct 16 19:00:01 Tomato syslog.info root: -- MARK --

    Oct 16 19:15:01 Tomato cron.info crond[621]: crond: USER root pid 5239 cmd rcheck --cron

    Oct 16 19:30:01 Tomato cron.info crond[621]: crond: USER root pid 5755 cmd rcheck --cron

    Oct 16 19:45:01 Tomato cron.info crond[621]: crond: USER root pid 6270 cmd rcheck --cron

    Oct 16 20:00:01 Tomato syslog.info root: -- MARK --

    Oct 16 20:00:01 Tomato cron.info crond[621]: crond: USER root pid 6786 cmd logger -p syslog.info -- -- MARK --

    Oct 16 20:00:01 Tomato cron.info crond[621]: crond: USER root pid 6788 cmd rcheck --cron

    Oct 16 20:04:15 Tomato daemon.warn dnsmasq-dhcp[1596]: Ignoring for DHCP host name W4254369

    Oct 16 20:15:01 Tomato cron.info crond[621]: crond: USER root pid 7302 cmd rcheck --cron

    Oct 16 20:30:01 Tomato cron.info crond[621]: crond: USER root pid 7808 cmd rcheck --cron

    Oct 16 20:30:18 Tomato daemon.warn miniupnpd[1318]: HTTP Connection from 192.168.1.35 closed unexpectedly

    As you can see nothing shows up in the logs. I am running Ad blocker. I tried shuting down most processes to see if that did anything but it seems random. I only have 20 mac addresses on my lan. Is there a debug mode I can set for more output to the log? The feature look great so far, keep up the good work!!!
     
  15. teaman

    teaman LI Guru Member

    @STEVEWEST - I haven't done much testing with K26 builds myself, but still...
    By the looks of the 'name' of this particular Tomato mod... I guess you're probably running a shibby20 build, right? I'd suggest taking a look at this thread (just in case):
    http://www.linksysinfo.org/index.ph...ncluded-bittorrent-client-beta-testing.33858/
    Fair enough. So it seems to work just fine at startup (or at least, for a while).
    Anything happening in-between those moments? WAN up, down or even reconnecting, by any chance? Any other hints and/or tips and/or clues about what might be /triggering/ such thing? Also, please notice there are a few different pages under the 'IP Traffic' menu... and even if the raw data (source) should/would be the same... keep in mind/consider it's read/gathered in a slightly different way:
    • Data presented on both Real-Time (ipt-realtime.asp) and Transfer Rates (ipt-details.asp) pages is obtained directly by reading raw data/stats provided by ipt_account (are any of these still working when those issues are noticed?).
    • However, data on the Last 24h (ipt-24.asp), Daily (ipt-daily.asp) and Monthly history pages (ipt-monthly.asp) is obtained via cstats... so I must ask: have you double-checked if the /path/ for the cstats datafile is accessible (readable? accessible? is it a CIFS and/or any kind of network share? is it a USB HD and/or flash drive? is it mounted/writeable? etc...)

    From your post, this seems to indicate ipt_account + cstats are actually working (rules being loaded, daemons running, collecting data/stats...):
    Code:
    root@Tomato:/tmp/home/root# iptables -nvx -L | grep -i account
    3249 1262086 all -- * * 0.0.0.0/0 0.0.0.0/0 account: network/netmask: 192.168.1.0/255.255.255.0 name: lan
    (...)
    root@Tomato:/proc/net/ipt_account# ps | grep stats
    (...)
    1585 root 1048 S cstats --new
    (...)
    Then....
    Which pages seem to stop working? Any/all pages under the IP Traffic menu? Or... just some of them?

    Anyways - back to your report (after these issues start happening):
    Code:
    (...)
    41060 10588605 all -- * * 0.0.0.0/0 0.0.0.0/0 account: network/netmask: 192.168.1.0/255.255.255.0 name: lan
    (...)
    1585 root 1072 S cstats --new
    (...)
    So, by looking/comparing some of that information gathered/reported 'before' and 'after' you noticed something strange was happening, here's a few possibly interesting/relevant remarks:
    1. 37811 packets did match that ipt_account rule (41060−3249), or 9326519 bytes (8.89MB) were accounted/forwarded thru (10588605−1262086). This part/piece seems to be (still) working as it should, therefore... the I'd expect both the Real-Time and Transfer Rates pages would also be fine (is this the case?)
    2. the cstats daemon was started with the '--new' option, which means any older/previous data/stats were lost/zero-ed (creates new/empty datafile)
    3. the cstats daemon was started only once and was still running (under the very same PID) when those issues started to happen - so no crash/restart - which is a good thing. As the daemon was still alive/running... that also shouldn't be 'the' problem
    4. cstats memory usage grew by 24KB (1072-1048), which probably means cstats had just 'learned' about 3 new IP addresses (est. around 8KB to keep stats for each address cstats V2 keeps history/track BW/IPTraffic usage)
    Considering the notes above... any new ideas/tests/checks/notes/ideas you could perhaps share so we could try to figure this one out? ;-)
    Thank you very much! :)
     
  16. STEVEWEST

    STEVEWEST Networkin' Nut Member

    Been try to experiment with different thing.

    1. When the GUI dies all menus stop data (graphs disappear, stats disappear and stop updating). IP traffic Last 24 Hours will show totals if I hit refresh, but it looks like they are total from with it lock up.

    2. Killing to process and starting it again does not help. Even restarting with a new file doesn’t help. Only a reboot bring the stats back for a while.

    3. Limiting the number of IP address it tracks didn’t help. One thing I noted is transfer Rates keeps track of all IP, even if Enable Auto-discovery is not checked.

    4. The rstat data is OK (graphs and data) , I assume you are using the same process to display them!

    5. WAN is not bouncing and have tried different storage locations (RAM, USB HD) both are accessable after.
     
  17. shibby20

    shibby20 Network Guru Member

    i havent any problems with IPT. Tested on RT-N16 with 079-BT-VPN. Router uptime - 13days.
     
  18. dingmel

    dingmel LI Guru Member

    Hi, Which version would work for WRT54G routers?

    Thanks
     

Share This Page