Is the rstats File Format Documented?

Discussion in 'Tomato Firmware' started by DW_Duck, Dec 16, 2008.

  1. DW_Duck

    DW_Duck Addicted to LI Member

    Is there any documentation for the format of the rstats file (other than reading the source code)? I have spent about an hour reading the source code and am hoping for something faster - maybe even just a pointer to the right *.h file would be a big help.

    Background if you care:

    I have little choice other than Hughes Net internet service with their nasty FAP (Fair Access Policy). Basically, download 200 MB without a rest and get shut off for 24 hours. It's more complicated than that, but that doesn't matter here.

    There is an open source FAP monitor (hnFAPMon) that seems to work pretty good, but I've seen bizarre behavior from it. I'd like to use the rstats data from tomato to check hnFAPMon and see that it's getting it right.

  2. mstombs

    mstombs Network Guru Member

    Its all self contained in the single source file rtstats.c, but guess you got that far?

    For an example of reading the web formatted file, with similar reason see the great script here:-
  3. tievolu

    tievolu Network Guru Member

    The 24hr rstats file (rstats-speed.js) looks something like this:

    speed_history = {
    'eth0': {
     rx: [720 comma separated values],
     rx_avg: 566968,
     rx_max: 22192748,
     rx_total: 408217212,
     tx: [720 comma separated values],
     tx_avg: 2221440,
     tx_max: 10577011,
     tx_total: 1599436920 },
    'eth1': {
     rx: [720 comma separated values],
     rx_avg: 2230480,
     rx_max: 13026628,
     rx_total: 1605946080,
     tx: [720 comma separated values],
     tx_avg: 591794,
     tx_max: 22496154,
     tx_total: 426092240 },
    'vlan0': {
     rx: [720 comma separated values],
     rx_avg: 3730,
     rx_max: 4194,
     rx_total: 2686024,
     tx: [720 comma separated values],
     tx_avg: 9613,
     tx_max: 17412,
     tx_total: 6921774 },
    'vlan1': {
     rx: [720 comma separated values],
     rx_avg: 534173,
     rx_max: 21832015,
     rx_total: 384605270,
     tx: [720 comma separated values],
     tx_avg: 2211826,
     tx_max: 10568762,
     tx_total: 1592515146 },
    'br0': {
     rx: [720 comma separated values],
     rx_avg: 2177930,
     rx_max: 10457785,
     rx_total: 1568110157,
     tx: [720 comma separated values],
     tx_avg: 574313,
     tx_max: 22314054,
     tx_total: 413505606},
    _next: 33};
    So you have a section like this for each interface:

    '[interface name]': {
     rx: [720 comma separated values in bytes],
     rx_avg: [value in bytes],
     rx_max: [value in bytes],
     rx_total: [value in bytes],
     tx: [720 comma separated values in bytes],
     tx_avg: [value in bytes],
     tx_max: [value in bytes],
     tx_total: [value in bytes]},
    Each of the 720 values in the RX and TX data represents bandwidth used in a two minute period - 720 values is 1440 minutes or 24 hours.

    You're interested in the RX data for vlan1 (the WAN interface). You can read the data in the following way (this is how I did it anyway):

    #!/bin/sh -x
    killall -USR1 rstats
    sleep 2
    while read line
      case $line in
      *vlan1*) read_next_rx=1 ;;
      if [[ $read_next_rx = 1 ]]
        case $line in
          *rx:*) rx_data=$line ; read_next_rx=0 ;;
    done < /var/spool/rstats-speed.js
    rm /var/spool/rstats-speed.js
    total_bytes=`echo $rx_data | awk '
        values_string = substr($0, index($0, "[") + 1, index($0, "]") - 6);
        split(values_string, values_array, ",");
        for (i=(721-number_of_values); i<=720; i++) {
        total += values_array[i];
        } printf("%0.f" total);
      }' number_of_values=$number_of_values total=0`
    echo expr $total_bytes
    Where $1 is an integer less than 720. So, for example, if you paste the code above into a script and run

    [script name] 210

    You'd get the total number of bytes downloaded over the last 7 hours, (210 = (7x60)/2).

    The only part of the data I don't understand is the "_next:" field at the bottom of the file. I'm not sure what that represents. It wasn't important for what I wanted to do so I didn't bother pursuing it.
  4. DW_Duck

    DW_Duck Addicted to LI Member

    Wow, that was quick! Thank you. Yes, I had read through rstats.c, but not close enough to really understand it and I was still thinking there might be a bunch more I had to know (in my quick reading I never noticed the file format). Anyways, knowing that it's basically all in that file helps a lot. Example scripts help a lot too. My work is practically done! Actually, that Virgin Media STM script is pretty cool. Maybe if I get ambitious with my week off of work for Christmas and 4 weeks off from school I'll work that into doing the same idea for Hughes Net - no promises though. I think my wife already has a lot planned for me.

    Thanks again,
  5. DW_Duck

    DW_Duck Addicted to LI Member

    HughesNet FAP Monitor Done But For the Testing

    I got tievolu's script, figured it out, figured out the rstats file, and wrote my own FAP monitor for HughesNet. Now, I just need a weekend where my wife isn't home to test it. (With HughesNet if you get FAP'ed you're 24+ hours without internet - can't take the chance if my wife is home).

    I'm 90% sure my FAP moniter will prevent me from getting FAP'ed even if I try. I'll publish and make an announcement when I'm 99% sure.

    BTW, the _next field in the rstats-speed.js is how many seconds until the stats are updated next by the rstats process.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice