1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

(Shibby/Toastman) Issues with the Device List Screen.

Discussion in 'Tomato Firmware' started by Rokair, Jul 11, 2016.

  1. Rokair

    Rokair Serious Server Member

    Hello,
    I don't post often but I am a long time user of both DD-WRT (probably since about 200 time frame) and Tomato firmware (probably since about 2002-2003 time frame).

    In recent years I have been mostly using Toastman's tomato for routers at my house and that of my mother's house. Some time ago I attempted to try one of the Shibby builds for the router that was at my Mother's house because the builds for the same router (Linksys E1000v2) from the Toastman releases became too large to actually fit on the router. However, while I did like several of the features of that release I found that the Devices listed on the Devices' screen were not always correct. It seems that when a device would leave the area and thus would no longer be connected instead of showing correctly that the device was NOT connected it was still listed to show the remainder of the IP Lease time with a weak connection. Then it often still shows after this with just a MAC address and a week connection. For this reason I only choose to use the Shibby releases for the routers that would not support the Toastman builds.

    More recently Toastman restructured his builds to be closer to the Shibby builds and I have found after upgrading my Linksys E4200 v1 router to the 509.3 release (and manually correcting all the settings) that this release now also has the bug described above (though this is not an issue in the 508.2 builds - nor any of the builds for at least several years prior).

    Obviously this does not affect the direct functions of the router but when one is attempting to monitor connections to a router/location this does add an obstacle.



    Thank you,
    Rokair
    Louisville, KY Area
     
    Techie007 likes this.
  2. david3

    david3 Addicted to LI Member

    For what it's worth, I've been running Toastman 1.28.0507 for over a year, and I've noticed the same thing. It only happens sometimes, though. The old wireless device will get "stuck." Sometimes clicking on it will release the entry, and sometimes it will be stuck there until I reboot the router.

    Getting ready to upgrade to 1.28.0508.2 soon.
     
  3. Jose C

    Jose C Serious Server Member

    I have the same behavior on my routers running shibby 132


    Sent from my iPad using Tapatalk
     
  4. Techie007

    Techie007 Serious Server Member

    I have noticed this bug on Tomato Shibby builds for the longest time too. Definitely on version 132 anyway. Can we please get this bug figured out and fixed? Again, I realize that coder man-hours are limited, so I would gladly donate a reasonable amount if it would help get this issue fixed. I don't have the skill to do it myself or I would.
     
  5. HowD131

    HowD131 Network Newbie Member

    I have noticed this as well using Shibby version 128. I am adding an observation to the above comments. I noticed that the listed Interface is not always correct; it is listed as br0 rather than eth1 yet the device is no longer active on the network.
     
  6. pegasus123

    pegasus123 Addicted to LI Member

    i have seen the same issue with Tenda N60, but not with WRT54GL. Not sure what the problem is.
     
  7. Elfew

    Elfew Network Guru Member

  8. jerrm

    jerrm Network Guru Member

    This is an issue with the drivers. wl reports the phantom macs as still associated. Not sure what can be done about it since the drivers are a black box. Someone with more wl command knowledge needs to chime in.
     
  9. koitsu

    koitsu Network Guru Member

    The Devices List feature of Tomato is quite possibly one of the most complex pieces of Tomato there is. It is a very disgusting, ugly, and badly-coded mess of JS, "fake ASP", and underlying C code -- all of which is uncommented. I'm talking about Toastman branch Toastman-RT-AC below. Folks "wanting this fixed" have opened the Pandora's Box Hornet Nest, so consider yourselves warned -- and if you really care about it, since this is an open-source project, you really have to do the digging yourself.

    Here's what I can tell you about the code:

    The JavaScript refers to variables that "tie everything together". This includes: DHCP leases from dnsmasq (though done in a very wonky way), ARP, and several wireless interface commands (specifically code that uses ioctl() to query the wireless driver; the wl command is not actually used).

    The C code, which is implemented into the Tomato webserver itself, gets its data from several places. I suggest looking at src/releases/router/httpd/devlist.c and wl.c for the "heavy lifting". Anything relating to "wireless" below can apply to multiple interfaces/radios (ex. wl -i {interface} rather than just wl), and is further made complicated if multiple BSSIDs are used by the user:

    * MAC addresses come from parsing /proc/net/arp and DHCP leases from dnsmasq. I suggest that anyone seeing "the wrong interface for a device" check /proc/net/arp on the router itself and see what the output shows. If it's wrong there, then that's that (someone probably has a very weird network configuration).

    * Wireless device MACs come from the WLC_GET_ASSOCLIST ioctl() command when given to a wireless interface. This is the equivalent of the wl assoclist command.

    * Wireless device RSSI comes from the WLC_GET_RSSI ioctl() command when given to a wireless interface. Its passed the MAC address of the individual device. This is the equivalent of the wl rssi {MAC} command. The data used from these results are the MAC address, and the RSSI value itself.

    * Wireless device TX/RX rate comes from the WLC_GET_VAR ioctl() command, subcommand sta_info, when given to a wireless interface. Its passed the MAC address of the individual device. This is the equivalent of the wl sta_info {MAC} command. The data used from these results are the TX rate, RX rate, and the number of seconds elapsed since the device was associated (sta_info_t struct field named "in"). There's also code around this part to handle WDS, which I'm not familiar with.

    Here's an example of what wl sta_info shows you -- why I'm showing this will become obvious in a moment:

    Code:
    root@gw:/tmp/home/root# wl sta_info F8:E0:79:57:F8:B1
    STA F8:E0:79:57:F8:B1:
             rateset [ 1 2 5.5 6 9 11 12 18 24 36 48 54 ]
             idle 14 seconds
             in network 2013 seconds
             state: AUTHENTICATED ASSOCIATED AUTHORIZED
             flags 0x613a: WME PS N_CAP
             tx pkts: 323
             tx failures: 0
             rx ucast pkts: 557
             rx mcast/bcast pkts: 11
             rate of last tx pkt: 1000 kbps
             rate of last rx pkt: 6000 kbps
             rx decrypt succeeds: 381
             rx decrypt failures: 0
    
    * The sta_info_t struct is "documented" in release/src-rt-6.x/include/wlioctl.h. There is also a field in this struct named "idle", which claims to be "time since data pkg rx'd from sta". This field isn't used by Tomato at all.

    Why I'm providing all this: folks now should have the shell commands available to "figure this out", or at least narrow things down more.

    I'm left with the impression that determining "when" a wireless device truly "leaves" the network is painful and difficult to determine reliably -- which doesn't surprise me at all given how "best-effort" 802.11 wireless is in the first place.

    However, the "idle" struct field might need to be implemented into the gigantic mess that is the JS code in attempt to determine if something truly is "still on the network" or not. I really don't know though if this is the correct approach; @jerrm stating that this is the wireless drivers' responsibility is absolutely 100% correct. But again, wireless is this really "best-effort" technology, so...

    If DD-WRT solved this problem, then odds are BrainSlayer knows how. It would be worthwhile to ask him how DD-WRT's Device List equivalent works, or look at the code yourself and see if a similar implementation/methodology could be used in Tomato.

    I'll add this final note: it is possible to disassociate a device with the BSS/AP manually. wl disassoc {MAC} will do that for you. So if you have something "stale", you might consider trying that and see if the device disappears from Tomato's Device List page (it might and it might not -- again, it's an amalgamation of DHCP leases, ARP, and the wireless driver data).
     
    Techie007, pawnu33 and Toastman like this.
  10. Toastman

    Toastman Super Moderator Staff Member Member

    From the OP's post, it seems that earlier versions of my firmware worked ok, and the issue was introduced after aligning with shibby's branch? So it ought to be possible to see some change that was responsible, if anyone has the time to do it? Personally, I don't see it as an important issue. Frankly, that information isn't of much use to me, and most routers don't display it for the same reason.
     
  11. Grimson

    Grimson Networkin' Nut Member

    Some pointers of this, as I tried to track this down in the past:

    This even happened with Teddy's Tomato builds and has been happening with Toastman builds as long as they existed, it's nothing new.

    It only happens with specific wlan adapter and driver combinations. For example I could reliably reproduce it with an old netbook, that was using a Ralink RT2860 card and Ralink's driver version "3.00.02". Using a common driver from Microsoft it didn't happen.

    So if it didn't happen for you and suddenly started it's most likely a new client device or an updated driver.

    I wasn't able to find why exactly the Broadcom driver tracks those devices as still connected, as I don't have the equipment to monitor the raw wlan traffic (meaning the actual radio chatter) and (like anyone else here) I lack access to the Broadcom driver sources.
     
  12. Jose C

    Jose C Serious Server Member

    That is what I wanted to add, it only happens with some devices, is not the wrong behavior for all the devices, for example my iPhone 6s disappear from the list as soon as I turn off the wifi on the phone


    Sent from my iPhone using Tapatalk
     
  13. ruggerof

    ruggerof LI Guru Member

    Just a stupid question, is your netbook really off or is it in standby? At home if I keep my MacBook Air in standby and my HP EliteBook in standby I can see them in the Device list as connected too, however if I really switch them off, they instantly disappear.
     
  14. pawnu33

    pawnu33 Network Newbie Member

    A quick question regarding device list. Sorry, i'm new to asp and all this. My question is how would i be able to get the result of command such as 'arp -a' , 'brctl show' and display it on the browser under certain div class?
    I see the shell command execution under tools-shell.asp like this.
    Code:
            function execute()
            {
                // Opera 8 sometimes sends 2 clicks
                if (cmd) return;
                spin(1);
                cmd = new XmlHttp();
                cmd.onCompleted = function(text, xml) {
                    eval(text);
                    updateResult();
                }
                cmd.onError = function(x) {
                    cmdresult = 'ERROR: ' + x;
                    updateResult();
                }
                var s = E('_f_cmd').value;
                cmd.post('shell.cgi', 'action=execute&command=' + escapeCGI(s.replace(/\r/g, '')));
                cookie.set('shellcmd', escape(s));
            }
    And updateResult() function.
    Code:
            function updateResult()
            {
                E('result').innerHTML = '<tt>' + escapeText(cmdresult) + '</tt>';
                cmdresult = '';
                spin(0);
            }
    I am not sure where the cmdresult is initialised or how the result of command execution can be stored. I don't quite understand what is happening here.. could anyone explain please.
     
  15. jerrm

    jerrm Network Guru Member

    cmdresult is global on the page. Best guess is it is being set from inside the eval(text) call.
     
    pawnu33 likes this.
  16. koitsu

    koitsu Network Guru Member

    As for "where" to get the output: try Tools -> System Commands. The output from there ends up in a <pre> with the ID selector of "result" (i.e. <pre id="result">). There is no containing <div>.

    Can I ask what you're trying to do?
     
    pawnu33 likes this.
  17. pawnu33

    pawnu33 Network Newbie Member

    I'm trying to create a device list from result of 'arp -a' and show them on the browser based on which bridge they're in.
    I have partially modified the GUI to enable isolation based on device/bridge and just require running relevant command (iptables/ebtables/wl command) on shell depending on user selection. I studied the device list section to replicate the list and as you said it's quite complex.
     
  18. koitsu

    koitsu Network Guru Member

    I will take the time to explain this step by step, but that's as much time as I'm willing to put into this.

    arp -a output is a less-verbose version of /proc/net/arp. The DNS resolution feature of the arp command is irrelevant (ARP has nothing to do with DNS).

    If you examine the code I linked earlier in this thread, specifically src/release/router/httpd/devlist.c function asp_arplist(), you will see that the data from that kernel proc is returned in the JavaScript variable called "arplist".

    The asp_devlist() function calls asp_arplist().

    asp_devlist()'s "function name" inside of the Tomato HTTP server, which offers a very "fake ASP" (think of it more like .shtml (server-parsed HTML)), is devlist. You can see this definition in src/release/router/httpd/tomato.c structure aspapi[].

    Therefore, the "fake ASP" call to this is: <% devlist(); %>.

    This call is made automatically by the following web pages -- the last one is what's relevant to this conversation:

    ipt-24.asp
    ipt-details.asp
    ipt-daily.asp
    ipt-24.asp
    ipt-realtime.asp
    ipt-monthly.asp
    status-devices.asp

    These are found within src/release/router/www and correlate directly with the pages that you can visit via the web GUI (e.g. http://192.168.1.1/status-devices.asp).

    Now that you know this: rather than trying to reinvent the wheel, I would suggest looking at the already-generated-by-Tomato JS variables that are available. You don't need to "call arp -a" or anything else -- the data is already there for you. Examine the JS variables for the status-devices.asp page and you'll find one called arplist, which is a multi-dimensional array with data such as:

    arplist[0][0] = "192.168.1.xx"
    arplist[0][1] = "aa:bb:cc:dd:ee:ff"
    arplist[0][2] = "br0"

    /proc/net/arp contains 3 other fields which are irrelevant to this situation (and Tomato does not export them): HW type, flags, and mask. The Linux proc(5) man page has details of what these mean, if you care (but again: irrelevant to this situation -- the data you're wanting is already there).

    Now given the above, you should be able to accomplish what you need in JavaScript/HTML as you wish.
     
    Last edited: Aug 26, 2016
    Techie007 and pawnu33 like this.
  19. pawnu33

    pawnu33 Network Newbie Member

    Thanks for your insight koitsu,
    I used the arplist and dhcpd_lease to create device list within status-devices.asp but i'd like to use the shell to run a list of commands to set up isolated wifi environment depending on user input..
    This test function fails to run the command for turning off wl0.3.
    Code:
            function initialsetup() {
               var command = "nvram set wl0.3_radio ='0'";
               var cmd = new XmlHttp();
               cmd.post('shell.cgi', 'action=execute&command=' + escapeCGI(command.replace(/\r/g, '')));
    
            }
    nvram still shows radio to be 1.

    My deadline is today midnight so i'd really appreciate any pointers. Thank you :oops:
     
  20. Edrikk

    Edrikk Network Guru Member

    When you say "deadline is midnight" is this for your work/employer?
    If so, I assume will publish your code update and fixes as a Pull Request back to Tomato?
     
  21. pawnu33

    pawnu33 Network Newbie Member

    It's for my school project and more of a modification of status-devices.asp page rather than fix/update.. by deadline i mean i'm rushing to get a prototype of my design done for a presentation. :p
     
  22. koitsu

    koitsu Network Guru Member

    I see and know what your problem is with this, and I'm sorry to say there are several problems (URL being POSTed to is not RFC compliant hence rejected; you do not understand HTTP forms -- while simultaneously I don't know what exec.cmd() really does under the hood so this point might be wrong (unlikely though), NVRAM command syntax is wrong, nvram command in question isn't actually going to do anything with the radio, etc.), none of which I am willing to go into. Sorry for sounding like a jerk, it's not my intention, but, from my previous post:

    If you want to run shell commands via the web GUI, I have talked about such in the past, though done from a different context (UNIX machine with curl, not within JS itself; the environments there differ).

    Good luck!
     
    pawnu33 likes this.
  23. Elfew

    Elfew Network Guru Member

  24. Techie007

    Techie007 Serious Server Member

    I've been noticing that this issue happens on a lot of routers (see the list below), but doesn't happen on the Tenda AC15. In fact, on the Tenda AC15, the device list is extremely accurate when showing connected wireless devices. Disconnected/out of range/roaming devices disappear 2–10 seconds after disconnecting, while sleeping devices stay solidly connected. Its accuracy is pretty much 100%. So we know it's possible, despite the "best effort" nature of the 802.11 standard.
    I've written a system that keeps track of connected devices, combining the wl assoclist and information from the dnsmasq leases file, and am confirming that this issue is caused by erroneous information from the wl assoclist command as detailed so well by @koitsu above. But this bug makes such a system nearly useless!

    The following routers definitely have the issue:
    • ASUS RT-N12 B1
    • ASUS RT-N12 D1
    • Linksys E1200
    • Linksys E1500
    • Tenda N80
    Any ideas to why these routers have the issue and the Tenda AC15 does not? The only major difference I can think of is that the Tenda AC15 is an ARM router, while the others are all MIPS routers.
     
  25. Techie007

    Techie007 Serious Server Member

    Just clarifying that the Tenda AC15 is running Tomato too (2017.3b13-kille72, with WL version 6.37.14.86). Is there any possibility that this Tomato bug could be fixed on other routers now that we know that it is possible for it to work correctly?
     

Share This Page