1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Useful One-Liners

Discussion in 'Tomato Firmware' started by Planiwa, Dec 17, 2009.

  1. Planiwa

    Planiwa LI Guru Member

    [Perhaps people will post their useful one-liners, and discuss them?]

    Mean Packet Size (Only works until 4.3 GB, because of 32-Bit counter reset!)

    Command:
    Code:
    echo "IF   RXPS TXPS  RX/TX-Bytes";ifconfig|awk '/Link/{IF=$1}; $2~/packets/{gsub("packets:","");P[$1]=.01+$2}; $2~/bytes/{gsub("bytes:","");B[$1]=$2; B[$5]=$6; print IF "\t" int(B["RX"]/P["RX"]), int(B["TX"]/P["TX"]) "\t" B["RX"],B["TX"]}'
    
    
    Output:
    Code:
    IF   RXPS TXPS  RX/TX-Bytes
    br0	520 761	1643273606 2232279871
    eth0	765 546	2286418108 1734774501
    eth1	534 768	1687497835 2256365810
    lo	87 87	127393 127393
    ppp0	736 521	2196538285 1651652826
    vlan0	0 153	0 547916
    vlan1	747 546	2232647752 1734226585
    
    
    The above was taken after several hours of significant uploading -- you can see that upload amounts to 75% of download. This is why the packet sizes have such uncharacteristic values.

    Later, just before the first 32-bit overflow, we got this:

    Code:
    IF   RXPS TXPS  RX/TX-Bytes
    br0	388 911	1712615457 4239129128
    eth0	10 414	48541381 1841439977
    eth1	402 917	1774570766 4277879633
    lo	87 87	171901 171901
    ppp0	888 390	4200637990 1724844128
    vlan0	0 164	0 1193646
    vlan1	897 415	4258113959 1840246331
    Of course, after the overflow the numbers become completely useless:

    Code:
    IF   RXPS TXPS  RX/TX-Bytes
    br0	376 71	1724158484 352608199
    eth0	91 402	462214471 1857880948
    eth1	390 79	1788620348 393951159
    lo	87 87	176964 176964
    ppp0	61 377	310226362 1736372444
    vlan0	0 166	0 1400441
    vlan1	73 402	371383591 1856480507
    
    The rxtx command uses the same ifconfig command as its source, but it is not affected by the 32-bit overflow, and it measures over a 1 hour interval:

    Code:
    Dec 18 18:59:03 ROUTER user.info RXTX: (IF MB B/P CMB)  ppp0 <385>7 {1462}49 [310]1737  eth1 <9>390 {62}1490 [1789]394
    Dec 18 19:59:03 ROUTER user.info RXTX: (IF MB B/P CMB)  ppp0 <388>7 {1469}48 [699]1743  eth1 <9>394 {62}1491 [1797]788
    Dec 18 20:59:03 ROUTER user.info RXTX: (IF MB B/P CMB)  ppp0 <220>4 {1465}52 [918]1748  eth1 <5>223 {63}1486 [1802]1011
    
    The packet sizes of about 50-60 upstream and just under 1500 downstream are typical of a heavy download pattern.
     
  2. Planiwa

    Planiwa LI Guru Member

    Another way to (show bytes and packets and) measure packet size

    Display (kilo-)packets and (mega-)bytes:

    Code:
    iptables -vxL FORWARD|awk '/ESTA/{print int($1/1000),int($2/1000000)}'
    
    If we just want to measure average packet size (average of upstream and downstream), we can do this:

    Code:
    iptables -vxL FORWARD|awk '/ESTA/{print int($2/$1)}'
    How about if we decide to use the shell to do the arithmetic?:

    Code:
    set -- $(iptables -vxL FORWARD|grep ESTA); echo $(($2/$1))
    or even:

    Code:
    set -- $(iptables -vxL FORWARD); while [ "${11}" != "RELATED,ESTABLISHED" ];do shift;done; echo $(($2/$1))
    This will work at first. But after about 2GB of traffic it will fail.

    Does anyone know why? :)

    It's always best to use awk for arithmetic, unless you know the arithmetic limitations of your particular shell implementation. Awk does high precision arithmetic.
     
  3. Elbart

    Elbart LI Guru Member

    My money's on 2^32.
     
  4. Planiwa

    Planiwa LI Guru Member

    Here's a hint:

    Code:
    # echo $((0-9876543210))
    -2147483647
    # 
    
     
  5. noodles2k

    noodles2k Addicted to LI Member

    it's the signed int maxval... what about showing data in unsigned Llongs?
    mine's on 2^31-1 :wink:
     
  6. Planiwa

    Planiwa LI Guru Member

    But why? Why is the ifconfig count limit 2^32 but the shell limit is 2^31? :)
     
  7. noodles2k

    noodles2k Addicted to LI Member

    I'd assume that in ifconfig the ints are unsigned and the limit for uints is 2^32-1. I'd guess that ash is automatically signing ints?
     
  8. noodles2k

    noodles2k Addicted to LI Member

    It's also possible ofcourse that ifconfig isn't outputting the numbers as ints, but as strings... either way it's down to the signed int limit. What about reading the numbers off as strings, converting them to hex, perform any calculations on the hex and then convert them back to strings?
     
  9. Planiwa

    Planiwa LI Guru Member

    Ifconfig gives counts. Natural numbers.
    Shell arithmetic is in Integers, which can be negative.

    So, the moment the shell does arithmetic, the quantity is limited to half the range of the values.

    So, even though Ifconfig can show values such as 3000111222 and 2000111222, the shell will surprise you when you try to subtract them:

    Code:
    # echo $((3000111222-2000111222))    
    147372425
    # 
    
    BTW, numeric comparison works properly:

    Code:
    # if [ 2222000111222 -lt 3333000111222 ];then echo less;fi
    less
    # if [ 2222000111222 -lt 1333000111222 ];then echo less;fi
    # 
    
     
  10. noodles2k

    noodles2k Addicted to LI Member

    I'm not certain it does. when numbers in ifconfig get to >2^32 do they become 0 again?

    It isn't limited to half the range as such, you still have exactly the same amount of data, but it's offset by -2147483647

    For the comparison, it uses unsigned big ints or longs - either way the max is 9223372036854775807
     
  11. mstombs

    mstombs Network Guru Member

    So here is an clear benefit of using a 64 bit OS

    Code:
    user@Ubuntu64bit:~$ echo $((0-9876543210))
    -9876543210
    user@Ubuntu64bit:~$ echo $((3000111222-2000111222))
    1000000000


    but it will still overflow/wraparound

    Code:
    user@Ubuntu64bit:~$ echo $((65535*65535))
    4294836225
    user@iUbuntu64bit:~$ echo $((4294836225*4294836225))
    -1125874137300991
     
  12. Planiwa

    Planiwa LI Guru Member

    The first post states this and illutrates it. :)

    That's why I wrote "quantity". :)

    I just noticed the mention of "unsigned Longs" in post #5.
    Can you do that in Tomato's shell?
     
  13. Planiwa

    Planiwa LI Guru Member

    In the case of Tomato, the big advantage of 64-bit would be that the ifconfig command would show useful counts.

    4.3 GB counters can fill up in a couple of hours on a typical Tomato router.

    One could circumvent the shell's limits with awk, but there is no substitute for ifconfig. Of course, one could change the busybox's ifconfig code to use Long Int's ...

    (At least iptables does that right. :)
     
  14. noodles2k

    noodles2k Addicted to LI Member

    so it does...

    doing
    Code:
    if [ 2222000111222 -lt 1333000111222 ];then echo less;fi
    
    uses unsigned longs, so they're available but I have no idea how to make use of them.
    "cat /proc/net/dev" ?
     
  15. mstombs

    mstombs Network Guru Member

    Imagine how fast a gigabit router would wrap around then! I recall a discussion a long time ago, but the Linux kernel devs refused to consider changing some 32 bit counters, they said it was up to a userspace app to poll as frequently as it needed to detect wraparound, I don't know if they have since seen sense...
     
  16. Planiwa

    Planiwa LI Guru Member

    This is a good equivalent to ifconfig -- easier to parse, but no more precise.
     
  17. Planiwa

    Planiwa LI Guru Member

    That's almost funny.

    Gigabit? That would be 35 seconds. :)
     
  18. noodles2k

    noodles2k Addicted to LI Member

    what about having a script that logs i++ to file every time there's a wrap around, and then another script that reads the ifconfig and appends 3.99*i onto the output when you call it?
     
  19. Planiwa

    Planiwa LI Guru Member

    That's a very interesting idea. It might be useful in conjunction with JFFS.
    I imagine using Unary, and appeding just 1 bit each for each 4.3GB rollover.

    it is probably not possible to just add 1 bit, but even one byte would be pretty good, for minimizing re-flashing.

    After a few days, the number of 4.3GB epochs may be enough precision.

    For those who want more granularity -- i.e. MB/hr, there is this:

    http://www.linksysinfo.org/forums/showthread.php?t=63480

    I'll post latest source of RXTX if there's interest (and a willingness to run it and share results.)

    (Instead of keeping track of # of rollovers it tracks the actual packets and bytes).
     

Share This Page