[Fork] FreshTomato-ARM

Discussion in 'Tomato Firmware' started by kille72, Apr 15, 2018.

  1. kernel-panic69

    kernel-panic69 Network Newbie Member

    Yes, 5GHz isn't supported out-of-the-box. There are a few instances of workarounds, I think, but I haven't dug into OpenWRT all that much.
     
  2. joksi

    joksi Serious Server Member

    Has anything new happened with 2,4ghz download performance on netgear r6250?
     

    Attached Files:

  3. TheHellSite

    TheHellSite New Member Member

    I just once again cleared all settings on my N18U and started from scratch.
    Configured only the basics, following this guide and applying the dns workaround. Time Service and VPN client finally works. At least for the AP itself. The clients however still can't connect to the internet, when having the AP set as gateway with or without vpn running. It seems that the routing table is messed up. While pinging a website directly from the AP works, doing the same on the client leads to "cannot resolve host...."

    BTW: while looking at the jffs page, it also gave me the error mentioned a few pages back. Seems that this errror kicks in after a few hours/days of operation?
     
  4. PetervdM

    PetervdM Network Guru Member

    i think you have to set the router as gateway, not the ap.
     
  5. TheHellSite

    TheHellSite New Member Member

    But how would my clients then be able to use the VPN Client of the AP? ;)
    Even without any service running on the it, the ap should redirect the traffic to my main router, when acting as the gateway for my clients. Which it doesn't!
     
  6. Mihai Olimpiu-Cristian

    Mihai Olimpiu-Cristian Networkin' Nut Member

    Some feedback on Xiaomi R1D 2018.3 build.
    I've upgraded (from Shibby 140) and don't know where along the way (configuring from scratch), I've lost internet access.
    Actually it seems no DNS requests are resolved. A movie is still streaming if the page is loaded prior to connecting to the router so there is a internet connection present and working.
    I checked the logs and the right DNS servers are received trough my PPPOE connection, no errors.
    Everything else seem to work.

    Downgrading to Shibby fixes the issue without resetting NVRAM.

    Notes on my config: I'm using an extra Vlan that is bridged to a second br1 bridge bound to a guest wireless connection so that different ip's get assigned for guests.
     
  7. Severus

    Severus Connected Client Member

    Hi guys, any progress on implementing dns over tls in tomato?
     
    Joe A, cyber062 and geekjock like this.
  8. rgnldo

    rgnldo Networkin' Nut Member

    Joe A likes this.
  9. moffa

    moffa Networkin' Nut Member

    Not sure if this is the right place to ask but I wanted to add haveged (an entropy generator) and dnscrypt-proxy (version 2) and needed a little help.
    I figured I need to add it to the router directory, but which scripts do I need to modify? Seems the Makefile, config file - anything else?
     
    rgnldo likes this.
  10. maurer

    maurer Network Guru Member

    why don't you use the entware haveged and dnscrypt-proxyv2 ?
     
  11. tvlz

    tvlz LI Guru Member

    Time to post this again :rolleyes:
    How To Start a New Thread & Why Should I
     
  12. Severus

    Severus Connected Client Member

    You think I should create a new topic just to ask about progress on implementing new function which was mentioned in this thread? :rolleyes: What should then be posted in this thread? Only instructions on How To Start New Thread? ;)
     
    Wizardknight, JoeDirte and Joe A like this.
  13. Cherkowski

    Cherkowski Network Newbie Member

    Just installed on DLink 868L vB this morning, happy I don't have to jump to ddwrt.

    None of my computers can see the Samba share, but no big deal I'm just using ftp for now.
     
  14. tvlz

    tvlz LI Guru Member

    YES, You asked today, somebody else will ask in a few more posts & again on the next page after the answer gets lost, if it was in it's own thread no getting lost.

    Thought I answered that in How To Start a New Thread & Why Should I
    Maybe you should post in that thread the reasons why starting a new thread is unwise from your point of view.;)
     
  15. rgnldo

    rgnldo Networkin' Nut Member

  16. rgnldo

    rgnldo Networkin' Nut Member

  17. JoeDirte

    JoeDirte Networkin' Nut Member

    Contrary to the thread police, that seems like a legit question for this thread. I wouldn't ask it myself, but I wouldn't fault anyone for asking about the next release in a thread about the releases. lol
     
    Joe A and apreslin like this.
  18. TaranScorp

    TaranScorp Network Newbie Member

    Will the FreshTomato firmware ever be able to be run on the Asus B2 HW ? Especially the Asus RT-AC68U-B2
     
  19. pomidor1

    pomidor1 Networkin' Nut Member

    Sory I was thinking about B1, it is produced B2?

    (yes, if you have a new OFW uploaded, you may not be able to upload the firmware via gui to bypass the lock you will be forced to use toamto restoration tool or if it fails, cfe mini web)
     
    Last edited: Sep 7, 2018
  20. TaranScorp

    TaranScorp Network Newbie Member

    I just got my router from Amazon. It came with B2 HW and 1.4 GHz manufactured in 2018, but can't install FreshTomato only Merlin's builds.
     
  21. TaranScorp

    TaranScorp Network Newbie Member

    googled toamto restoration tool , no such thing. I already tried the mini-web and no go , the b2 hardware is different from the rest of the hard-ware versions I think.
     
  22. bjlockie

    bjlockie Network Guru Member

  23. TaranScorp

    TaranScorp Network Newbie Member

    FreshTomato only supports AC68U(A1,A2,B1) no B2 for now. And mine was bought with a 1.4 GHz sticker on the front of box. I wonder if they will implement the B2 in future Firmwares?

    Thanks All
     
  24. kille72

    kille72 LI Guru Member

  25. Joe A

    Joe A Networkin' Nut Member

    Last edited: Sep 10, 2018
    rgnldo and kille72 like this.
  26. rgnldo

    rgnldo Networkin' Nut Member

  27. Pasha_ZZZ

    Pasha_ZZZ Serious Server Member

    Added lsusb & reboot into wanup script. And logging to JFFS.
    Code:
    Sat, 08 Sep 2018 03:03:43 +0300
    lsusb:
    Bus 001 Device 001: ID 1d6b:0003
    Bus 002 Device 001: ID 1d6b:0002
    No /dev/sda exists. Rebooting...
    Sat, 08 Sep 2018 03:06:05 +0300
    lsusb:
    Bus 001 Device 001: ID 1d6b:0003
    Bus 002 Device 001: ID 1d6b:0002
    Bus 001 Device 002: ID 174c:5106
    Sun, 09 Sep 2018 03:03:47 +0300
    lsusb:
    Bus 001 Device 001: ID 1d6b:0003
    Bus 002 Device 001: ID 1d6b:0002
    No /dev/sda exists. Rebooting...
    Sun, 09 Sep 2018 03:06:11 +0300
    lsusb:
    Bus 001 Device 001: ID 1d6b:0003
    Bus 002 Device 001: ID 1d6b:0002
    Bus 001 Device 002: ID 174c:5106
    Mon, 10 Sep 2018 03:03:56 +0300
    lsusb:
    Bus 001 Device 001: ID 1d6b:0003
    Bus 002 Device 001: ID 1d6b:0002
    No /dev/sda exists. Rebooting...
    Mon, 10 Sep 2018 03:06:19 +0300
    lsusb:
    Bus 001 Device 001: ID 1d6b:0003
    Bus 002 Device 001: ID 1d6b:0002
    Bus 001 Device 002: ID 174c:5106
     
  28. TaranScorp

    TaranScorp Network Newbie Member

    tell me this biggest launch will support the B2 hardware
     
    WillyTP likes this.
  29. rgnldo

    rgnldo Networkin' Nut Member

  30. churles

    churles Serious Server Member

    Last edited: Sep 11, 2018
    kille72 and pedro311 like this.
  31. CBR900

    CBR900 Network Guru Member

  32. rgnldo

    rgnldo Networkin' Nut Member

    Joe A and kille72 like this.
  33. tripper22

    tripper22 Serious Server Member

    I know it's just preliminary support for Stubby, but is there any way to use DNS over TLS with 2018.4?

    Like rgnldo said above thanks to @AndreDVJ @kille72 @pedro311 @Edrikk
     
    kille72 likes this.
  34. kille72

    kille72 LI Guru Member

    Stubby' is an application that acts as a local DNS Privacy stub resolver (using DNS-over-TLS).

    I don't understand the question...
     
  35. TaranScorp

    TaranScorp Network Newbie Member

    Why can't I install FreshTomato on my Asus RT-AC68U-B2?
    I'll never get to try it out. Merling installs fine though.
     
  36. suolin

    suolin Networkin' Nut Member

    Sorry if this was asked before (the search did not return any results) - are there any plans to support the Netgear R8500 in the future?
     
  37. churles

    churles Serious Server Member

    Just installed 2018.4 on my R7000. So far so good.
     
    kille72 likes this.
  38. Techie007

    Techie007 Serious Server Member

    Thank you for the continued development. So far, no apparent regressions. QoS is working as usual, as is Virtual WiFi. The main existing bug I've noticed (DNS lookups not working when used as an access point) remains, although this code, placed in the Scripts -> Init section, works around it:
    Code:
    sleep 10
    rm /tmp/etc/resolv.conf
    echo nameserver `nvram get wan_dns` > /tmp/etc/resolv.conf

    One thing I noticed, starting with 2018.3 and continuing with this version, there's a new Tomato icon at the top right, that doesn't appear unless I erase NVRAM after upgrading. So new routers that I'm flashing all have it, while routers that have been running Fresh Tomato since 2018.2 (and older) and been upgraded without resetting, don't show the new icon:
    [​IMG]
     
    Last edited: Sep 12, 2018
  39. user17600

    user17600 Reformed Router Member

    Adblock error, repeated log entry. Cold reboot did not help. Reverting to 2018.3.

    Sep 11 21:20:31 daemon.crit dnsmasq[6603]: error at line 47567 of /etc/dnsmasq.adblock
    Sep 11 21:20:31 daemon.crit dnsmasq[6603]: FAILED to start up

    EDIT: NVRAM=tried both ways - dirty and fresh. Router: R7000.

    Sorry for the oversight.
     
    Last edited: Sep 12, 2018
  40. kille72

    kille72 LI Guru Member

    Have you cleared NVRAM?
     
  41. Pasha_ZZZ

    Pasha_ZZZ Serious Server Member

    JFFS error for me appears only on formatting. JFFS formatting always ends with error, but after reboot JFFS works... Seems as fine.
     
    kille72 likes this.
  42. rgnldo

    rgnldo Networkin' Nut Member

    All reports of problems should be considered after the physical reset procedure of the router. The 2018.4 build has undergone many changes and updates. Do the hard physical reset of the router and reconfigure from the beginning. Do not forget to disable the JFFS partition before the physical reset.
    [​IMG]
     
    M_ars and pedro311 like this.
  43. rgnldo

    rgnldo Networkin' Nut Member

    @kille72 @pedro311 I suggest topics like this, based on the Kong builds, the DD-WRT forum

     
  44. Pasha_ZZZ

    Pasha_ZZZ Serious Server Member

    And what the difference between physical reset and 2nd option of webface reset ("Erase all data in NVRAM")? I think they do the same...
     
  45. tstrike34

    tstrike34 New Member Member

    Hey folks... Thank you so much for continuing this development. I upgraded to 9-11-18 release for R7000 Nighthawk router. Now I am seeing continous high CPU utilization 54% ... I did not wipe the NVRAM when I upgraded.

    Would wiping the NVRAM bring that CPU utilization down?
     
  46. rgnldo

    rgnldo Networkin' Nut Member

    I installed the 2018.4 build on the Asus AC68U, with physical reset. Very stable.
    DNS over TLS functional. Tested
    [​IMG]

    [​IMG]
     
    Joe A likes this.
  47. tripper22

    tripper22 Serious Server Member

    How does one install Stubby to use DNS-over-TLS?
     
  48. Techie007

    Techie007 Serious Server Member

    I am confirming this bug, and yes, I've cleared NVRAM (new router). Not sure if it's a regression or not since I've never used this feature before. It occurred at line 56079 in my case. Disabling Adblock returns the router to normal operation.
     
    user17600 likes this.
  49. Joe A

    Joe A Networkin' Nut Member

     
  50. TaranScorp

    TaranScorp Network Newbie Member

    What hardware version are you guys installing on the AC68U(A1,A2,B1) on my B2 there is a no go for version B2, no matter what I try.
     
  51. rgnldo

    rgnldo Networkin' Nut Member

    A1
     
  52. rgnldo

    rgnldo Networkin' Nut Member

    Stable for me
     
  53. Johan Van Dyck

    Johan Van Dyck New Member Member

    Hi

    I'm using "dir868l rev A" hardware with 2018.3 firmware. I configured "Tomato Update Notification System". I know a new version of freshtomato for my router is available (2018.4). However I don't get any update notification in the overview screen. Is this a bug?

    Kind regards

    Johan.
     
  54. TaranScorp

    TaranScorp Network Newbie Member

    Is there going to be a fix so FreshTomato can be installed on B2 Hardware?. On first post reads works on AC68U(A1,A2,B1) no B2 :(
     
  55. kille72

    kille72 LI Guru Member

    Check line 56079 in /etc/dnsmasq.adblock. I'm using AdBlock without any problems.
     
    Techie007 likes this.
  56. rgnldo

    rgnldo Networkin' Nut Member

    Stable for me

    [​IMG]
     
  57. rgnldo

    rgnldo Networkin' Nut Member

  58. koolershaker

    koolershaker New Member Member

    I made a hard reset with the pushpin on the backside of the router (Asus RT-AC68R/U)

    Configurated from Scratch and when i activate Adblock after a few minutes it starts this:

    Sep 13 15:51:37 unknown daemon.crit dnsmasq[6279]: FAILED to start up
    Sep 13 15:51:38 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:39 unknown daemon.crit dnsmasq[6282]: error at line 104443 of /etc/dnsmasq.adblock
    Sep 13 15:51:39 unknown daemon.crit dnsmasq[6282]: FAILED to start up
    Sep 13 15:51:40 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:41 unknown daemon.crit dnsmasq[6285]: error at line 104443 of /etc/dnsmasq.adblock
    Sep 13 15:51:41 unknown daemon.crit dnsmasq[6285]: FAILED to start up
    Sep 13 15:51:42 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:43 unknown daemon.crit dnsmasq[6288]: error at line 104443 of /etc/dnsmasq.adblock
    Sep 13 15:51:43 unknown daemon.crit dnsmasq[6288]: FAILED to start up
    Sep 13 15:51:44 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:45 unknown daemon.crit dnsmasq[6291]: error at line 104443 of /etc/dnsmasq.adblock
    Sep 13 15:51:45 unknown daemon.crit dnsmasq[6291]: FAILED to start up
    Sep 13 15:51:46 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:47 unknown daemon.crit dnsmasq[6294]: error at line 104443 of /etc/dnsmasq.adblock
    Sep 13 15:51:47 unknown daemon.crit dnsmasq[6294]: FAILED to start up
    Sep 13 15:51:48 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:49 unknown daemon.crit dnsmasq[6302]: error at line 104443 of /etc/dnsmasq.adblock
    Sep 13 15:51:49 unknown daemon.crit dnsmasq[6302]: FAILED to start up
    Sep 13 15:51:50 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:52 unknown daemon.crit dnsmasq[6305]: error at line 104443 of /etc/dnsmasq.adblock
    Sep 13 15:51:52 unknown daemon.crit dnsmasq[6305]: FAILED to start up
    Sep 13 15:51:53 unknown user.debug preinit[1]: dnsmasq terminated unexpectedly, restarting.
    Sep 13 15:51:54 unknown daemon.crit dnsmasq[6313]: error at line 104443 of /etc/dnsmasq.adblock
     
    Techie007 likes this.
  59. rgnldo

    rgnldo Networkin' Nut Member

    For efficiency in setting up DNS over TLS, for me, it looks like this:

    step 01

    [​IMG]

    step 02

    Dnsmasq
    Custom configuration:

    Code:
    bogus-priv
    no-resolv
    no-poll
    cache-size=0
    log-async=25
     
    Haldi4803, pajarillo and geekjock like this.
  60. tripper22

    tripper22 Serious Server Member

    @rgnldo How do you install Stubby to use DNS over TLS? TIA
     
  61. Severus

    Severus Connected Client Member

    It's baked into 2018.4 release...
     
  62. Johan Van Dyck

    Johan Van Dyck New Member Member

    --> today, 13/9/2018, the message appeared "Newer version of FreshTomato 2018.4 is now available (...)"
     
  63. tripper22

    tripper22 Serious Server Member

    @Severus Where is it? How do I enable it? Thanks.
     
    Johan Van Dyck likes this.
  64. Techie007

    Techie007 Serious Server Member

    I reenabled Adblock, and now the crash is happening at line 55448. I copied the file to a USB drive and opened it. It looks like there's a Unicode character on that line:
    55447 address=/secoptim.com/0.0.0.0
    55448 address=/secret.É¢oogle.com/0.0.0.0
    55449 address=/secretlanguage.co/0.0.0.0


    People are asking about where it is or how to enable it because not all routers appear to have it. Perhaps these routers don't have enough flash space to include Stubby? Here's a Tenda AC15 router running FreshTomato 2018.4, and as you can see, the options under Enable DNSSEC (in the LAN section) seen in @rgnldo's post don't show:
    [​IMG]
     
  65. rgnldo

    rgnldo Networkin' Nut Member

    Your router probably does not support some features. Its firmware build is only 12mb. It is not just Stubby, they are furnishing dependencies and libraries needed.
     
  66. CBR900

    CBR900 Network Guru Member

    Hi
    Can you help please... Im having problem with:

    CPU Usage 52% with ver 2018.4 and Asus AC68U

    please
     
  67. Johan Van Dyck

    Johan Van Dyck New Member Member

    Hi
    I guess not all features are indeed compiled. But it's hard to tell which feature in in which release. I'm using the dir-868l router (freshtomato-DIR868L-ARM-2018.4-special.trx release of 11.7Mb). My router's overview tells me I have 256Mb of flash and the filesystem is barely used.
    Filesystem Size Used Available Use% Mounted on
    /dev/root 9.4M 9.4M 0 100% /
    devtmpfs 124.8M 0 124.8M 0% /dev
    tmpfs 124.8M 2.6M 122.3M 2% /tmp
    devfs 124.8M 0 124.8M 0% /dev
    /dev/sda1 465.6G 147.1G 318.5G 32% /tmp/mnt/sda1

    So, Kille72, could you explain your build and why there is no room for more features?
     
  68. tripper22

    tripper22 Serious Server Member

    Okay Stubby (DNS to TLS) is only in the AIO version of the firmware not the VPN version.

    How do you test to make sure it is working properly?
     
  69. rgnldo

    rgnldo Networkin' Nut Member

    Build AIO
     
  70. rgnldo

    rgnldo Networkin' Nut Member

    [​IMG]
    On AC68U
     
  71. thyestes

    thyestes Network Newbie Member

    I upgraded last night, did a hard reset like rgnldo suggested and cleared NVRAM. The only error so far is this one with ADblock. I have a R7000 as well.

    Is there a solution for that?

    -edit- Did additional troubleshooting. In my case the list someonewhocares hosts zero hosts is causing the trouble. When I disable that list adblock works fine.
     
    Last edited: Sep 14, 2018
    Techie007 likes this.
  72. kille72

    kille72 LI Guru Member

    Good rule that we have previously talked about. Are you experiencing problems with FreshTomato? Before you report it, delete NVRAM and set it all manually, not from saved configuration!
     
    Goggy and rgnldo like this.
  73. CBR900

    CBR900 Network Guru Member

    Here I have erased nvram before installing and after the installation and setup everything manaully.

    Is there anther way to erase it totally?


    Sent from my iPhone using Tapatalk
     
  74. rgnldo

    rgnldo Networkin' Nut Member

    All reports of problems should be considered after the physical reset procedure of the router. The 2018.4 build has undergone many changes and updates. Do the hard physical reset of the router and reconfigure from the beginning. Do not forget to disable the JFFS partition before the physical reset.
    [​IMG]
     
  75. PetervdM

    PetervdM Network Guru Member

    the reset button is not always performing as expected. this is a good alternative:
    • make the router is as idle as possible.
    • goto "tools","system commands"
    • type sync;nvram erase;halt
    • press execute
    • wait a few seconds
    • switch the router off for at least 10 seconds
    • switch the router back on
    • leave it alone for 30 minutes!
     
    AndreDVJ, kille72 and rgnldo like this.
  76. rgnldo

    rgnldo Networkin' Nut Member

    Very good tip
     
  77. user17600

    user17600 Reformed Router Member

    I reloaded 2018.4 and looked at dnsmasq.adblock and can confirm that the error-reported line also seems to contain a unicode character.

    To the developers: first again my appreciation for keeping Tomato going. Can you comment on why 2018.4 adblock crashes when 2018.3 adblock, with the same exact lists, does not?

    Adblock is part of my network protective services and thus I cannot upgrade if it fails.

    I tried to add the unicode website to the whitelist but apparently unicode characters don't necessarily copy. Otherwise I am at a loss how to address this bug in 2018.4 (I am not a coder of any sort at all).

    Regards,
     
    Techie007 likes this.
  78. bodnid

    bodnid Network Newbie Member

    Guys, does this supersede Shibby's AdvancedTomato?
     
  79. Wizardknight

    Wizardknight Serious Server Member

    It is newer with more fixes & updates. So one could make that argument.
     
    bodnid likes this.
  80. Wizardknight

    Wizardknight Serious Server Member

    Feedback on a 6300v2 with 2018.4.
    So far so good.
    The 2.4ghz speed seems improved, but is still no where close to the bandwidth of 5Ghz.
    Maybe related to the driver update noted in the change log?
     
  81. bodnid

    bodnid Network Newbie Member

    Excellent, I'll update shortly.

    Does Wireless Ethernet Bridge work ok on this though?
     
  82. AndreDVJ

    AndreDVJ LI Guru Member

    @user17600 can you share your blocklist? I could not reproduce the issue using the default ones.
    Code:
    http://winhelp2002.mvps.org/hosts.txt
    http://adaway.org/hosts.txt
    http://hosts-file.net/ad_servers.txt
    http://www.malwaredomainlist.com/hostslist/hosts.txt
    http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
    https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt
     
    rgnldo likes this.
  83. geekjock

    geekjock Network Guru Member

    Thanks for sharing this. Would you please explain what the Dnsmasq settings do, and why they are needed?
     
  84. user17600

    user17600 Reformed Router Member

    Code:
    http://winhelp2002.mvps.org/hosts.txt
    http://winhelp2002.mvps.org/hosts.txt
    http://adaway.org/hosts.txt
    http://hosts-file.net/ad_servers.txt
    http://www.malwaredomainlist.com/hostslist/hosts.txt
    https://raw.github.com/notracking/hosts-blocklists/master/hostnames.txt
    http://mirror1.malwaredomains.com/files/domains.txt
    http://pgl.yoyo.org/as/serverlist.php?hostformat=nohtml
    https://www.dshield.org/feeds/suspiciousdomains_Medium.txt
    https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn/hosts
    https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
    https://hosts-file.net/hphosts-partial.txt
    http://malwaredomains.lehigh.edu/files/immortal_domains.txt
    
    As was noted above, it seems that having a unicode character causes the error in 2018.4 whereas the same domain causes no issue in 2018.3.

    I believe it is this entry:
    Code:
    address=/bireysel-z▒raat.com/0.0.0.0
    Note: this same domain is present in the 2018.3 list and there is no error. The above is actually copied from 2018.3.

    Edit: the unicode appears to be %u2592, UnEscape, in this particular case. No idea if there are other unicode domains in the list. So my question is on handling this issue in 2018.4 vs 2018.3.
     
    Last edited: Sep 15, 2018
  85. pajarillo

    pajarillo Network Newbie Member

    I'm also having trouble with adblock and dnsmasq

    just upgraded from Advanced Tomato (last version) in an Asus RT-N18U (erasing nvram after that). so far so good, only this issue.

    EDIT: It seems to be some entry from this list:

    I can't paste URL :/
     
    Last edited: Sep 15, 2018
  86. koitsu

    koitsu Network Guru Member

    For those having the "error at line X in /etc/dnsmasq.adblock" message described above:

    Tomato does not support what is called IDN. It probably never will. dnsmasq, thus Tomato, only understands ASCII domain names.

    The problem is either one or more of the Adblock blocklist servers in the list is returning either:

    a) A Unicode domain name -- except the Unicode glyph U+2592 would not be a valid Unicode domain name. U+2592 is a VT100-like character for "medium shading" (think old DOS graphics),

    or,

    b) Corrupted data (read: the adblock server returning that domain name has a problem), or a character set conversion problem on the server itself.

    (b) is the most likely explanation. My gut feeling is that the adblock server has been given an IDN, but the server software itself does not understand Unicode, and converted it into the wrong format. To find this out, we need to know what server is returning this data.

    So let's actually do some troubleshooting, yes? Some of the domain contains non-Unicode characters, so we can look for those by downloading all of the above links and grepping for that portion. egrep -r 'bireysel.*at.com' should do the trick.

    And here we find the problem server:

    https://hosts-file.net/hphosts-partial.txt

    This server returns the following entry:

    Code:
    127.0.0.1       bireysel-zîraat.com
    
    A hex dump of this line:

    Code:
    00000000: 3132 372e 302e 302e 3109 6269 7265 7973  127.0.0.1.bireys
    00000010: 656c 2d7a ee72 6161 742e 636f 6d0d 0a    el-z.raat.com..
    
    The offending byte is hexadecimal value 0xEE, which is not a Unicode character at all, it's a ISO-8859-1 (Latin-1) character set character. This is an invalid character in a domain name. The UTF-8 equivalent would have been bytes 0xC8 0xEE.

    When using the wrong character set for display on a terminal, you will get something completely different, such as U+2592. You have to understand character sets and byte encodings to know how to recognise such mistakes. I do not have the time to train people on that. :) The reason the character shows up OK on the forum (when I pasted it) is because I forced ISO-8859-1 encoding, and my terminal program was able to do the UTF-8 translation for me. This is why looking at the raw hex data is important!

    Solution: Someone needs to contact whoever maintains that adblock blocklist server and tell them of this mistake.

    Workaround: Remove https://hosts-file.net/hphosts-partial.txt from the blocklist.

    Hope this helps.
     
  87. AndreDVJ

    AndreDVJ LI Guru Member

    It's probably a change in Dnsmasq which made it even more sensitive to "bad" configuration files.

    I think I know how to fix it, but requires that I compile a firmware because adblock baked in the firmware won't easily run as a stand-alone shell script and I'm not willing to figure out what are the parameters required.

    I'm compiling an image. If works (I hope) I'll push the change to my repo.

    EDIT: Nup didn't work - I'll keep trying...
     
    Last edited: Sep 15, 2018
  88. koitsu

    koitsu Network Guru Member

    Possibly -- I wouldn't know about that. But speaking more generally:

    dnsmasq was never built with IDN support. And don't change that! Please keep it no-IDN as dnsmasq --version shows! The character in question isn't part of IDN, it's some botched character set entry that was permitted by whoever maintains that blocklist/server. The maintainer of that needs to fix their mistake. They probably have a website that lets you submit entries, and they are using ISO-8859-1 instead of UTF-8 like they should be.

    The "proper" long-term fix for this in Tomato would be: when aggregating all the hostnames together, strip out any lines that contain characters OTHER THAN regex [A-Za-z0-9\-] (i.e. capital A through Z, lowercase a through z, zero through nine, and hyphen) in the domain name portion. Just silently ignore/delete the lines for safety. Read this section of Wikipedia for that, specifically about Punycode (it's the only way IDN works with non-IDN things. Now you know what blocklist entries bireysel---subemiz--giris--tr.com and the like are for!).
     
    rgnldo, kille72 and pajarillo like this.
  89. AndreDVJ

    AndreDVJ LI Guru Member

    Yup that's what I'm trying to do - having a regex to ignore these lines so they won't crash Dnsmasq anymore :)
     
    rgnldo, M_ars, kille72 and 1 other person like this.
  90. Pasha_ZZZ

    Pasha_ZZZ Serious Server Member

    This regexp cleans list
    Code:
    cat hphosts-partial.txt|/usr/bin/tr -d '\r'|/bin/grep -Ei '\s[0-9a-z\.-]*$'|grep -Ev '^\s*#'
    Invert regex shows this
    Code:
    127.0.0.1    bireysel-zîraat.com
    127.0.0.1    bluewin_login_accountsmail.godaddysites.com
    127.0.0.1    h_t_t_p_s.facebook.com.3s3s.ru
    127.0.0.1    h_t_t_p_s.www.facebook.com.3s3s.ru
    127.0.0.1    microsoft_excel.de.downloadastro.com
    127.0.0.1    orangefrofxmailsf_inbox.godaddysites.com
    127.0.0.1    orange_identifiezvous1.godaddysites.com
    127.0.0.1    outlookupdate_team.editor.multiscreensite.com
    127.0.0.1    www.h_t_t_p_s.facebook.com.3s3s.ru
    127.0.0.1    www.h_t_t_p_s.www.facebook.com.3s3s.ru
     
    koitsu likes this.
  91. rgnldo

    rgnldo Networkin' Nut Member

    I have not used the Tomato lists for a while. I use only two sources, with more than 80,000 suspicious domains.
    [​IMG]
     
  92. koitsu

    koitsu Network Guru Member

    Thanks for this!

    The first entry is the one I covered. There are several entries here which are also bogus/bad (on the part of whoever is maintaining this blocklist server etc.), but be sure to read everything I say (esp. the bottom of my post) before excluding them:

    Code:
    127.0.0.1    bluewin_login_accountsmail.godaddysites.com
    127.0.0.1    h_t_t_p_s.facebook.com.3s3s.ru
    127.0.0.1    h_t_t_p_s.www.facebook.com.3s3s.ru
    127.0.0.1    microsoft_excel.de.downloadastro.com
    127.0.0.1    orangefrofxmailsf_inbox.godaddysites.com
    127.0.0.1    orange_identifiezvous1.godaddysites.com
    127.0.0.1    outlookupdate_team.editor.multiscreensite.com
    
    These are bogus/bad/invalid because they contain underscore in the hostname portion ("furthest left portion") of the FQDN.

    Note that there is a difference between hostname and "subdomain" (officially called a "label" or "binary label" in several RFCs) -- underscore is a permitted character in subdomains (which is how things like SRV records work). In fact, subdomains can, as I understand it, contain any characters given how RFC2181 outlines it. But underscores in the hostname portion are invalid.

    Which leads me to the last two:
    Code:
    127.0.0.1    www.h_t_t_p_s.facebook.com.3s3s.ru
    127.0.0.1    www.h_t_t_p_s.www.facebook.com.3s3s.ru
    
    ...which are actually valid FQDNs. The underscore is within the subdomain ("label"), while the hostname ("www") is also valid, in addition to the domain (3s3s.ru) being valid as well. I still think these are probably "wonky" in some way, though, i.e. are invalid submissions by some human/person that whoever maintains that hostlist erroneously accepted.

    Isn't parsing DNS records fun?

    Here's the ruse:

    A lot of these FQDNs actually exist in real DNS namespace. Meaning, DNS resolvers and DNS servers are actually allowing these entries, and thus they can in fact be resolved, despite their lack of RFC adherence. That means that there potentially are, in the wild, sites/etc. that are using these FQDNs for ads despite lack of RFC compliance. Let's try a couple:

    Code:
    $ host bluewin_login_accountsmail.godaddysites.com
    bluewin_login_accountsmail.godaddysites.com has address 198.71.232.10
    
    $ host outlookupdate_team.editor.multiscreensite.com
    outlookupdate_team.editor.multiscreensite.com has address 52.72.79.100
    outlookupdate_team.editor.multiscreensite.com has address 34.224.138.214
    
    $ host microsoft_excel.de.downloadastro.com
    microsoft_excel.de.downloadastro.com has address 108.163.213.235
    
    Yikes.

    So to be on the safe side, Tomato-wise, we should permit underscore (_) in the regex as well. :(

    Yes, this means bogus/completely invalid entries like "foo.bar_blat.com" (you definitely cannot have underscore in a domain name) would be allowed, but that's a trade-off to make the process easier. I wouldn't want some aggregation script having to parse hostnames vs. subdomains/labels vs. domain names -- that's overkill and not worth it (who should be doing that are the blacklist server providers that accept these entries, not Tomato! So shame on them for allowing bogus stuff). As much as I'd like everyone to be RFC compliant, the realities are resolvers seem to permit this, so we had best do so as well.

    Someone, again, should submit these to whoever maintains that blacklist. It seems they need to do some housekeeping (sanitation) and better adherence to RFCs for user-submitted entries. Lack of such is how that bireysel-zîraat.com entry got allowed in the first place. :(
     
  93. AndreDVJ

    AndreDVJ LI Guru Member

    I managed to get this done though that was annoying - grep/awk/sed and friends aren't my thing: https://bitbucket.org/AndreDVJ/advancedtomato-arm/commits/99b99800bbbf0ab0ef8f6e0a661145755738eae6

    The bulk of hosts files were concatenated and then redirected to some file (declared as WORK1). Now they get piped into sed first - to nuke anything that is not an ASCII character (0x00-0x7F) - and finally resulting output gets redirected into that file.

    I still don't call this as a solution. While I don't think plenty of valid ASCII characters are accepted by Dnsmasq, I don't think anyone will mess up hosts files that badly.
     
    Techie007, M_ars, user17600 and 3 others like this.
  94. RMerlin

    RMerlin Network Guru Member

    A few years ago I compiled it with IDN support, for test purposes. Then I decided the bloat wasn't worth it, and I removed it. It was making dnsmasq notably fatter.
     
    kille72 and koitsu like this.
  95. Pasha_ZZZ

    Pasha_ZZZ Serious Server Member

    What about idn from libidn? With underscire-permitted regex we can add remaining lines to filter after idn processing.
     
  96. koitsu

    koitsu Network Guru Member

    The idn(1) utility relies on libidn, which relies on gettext (libintl and usually pulls in libasprintf). Both libidn and gettext are quite fat/large: libidn is around 250KBytes, while gettext/libintl is smaller. However, in both cases both libraries include large sets of locale definitions to understand all of the possible character sets/locales (usually via .mo files); these are required for the tools to work properly. These .mo files seem small enough (9-12KBytes each), but there are many and they add up very fast. A program built with libidn and gettext support requires even more code and related bits too, so programs using them grow in size as well. In the end, I would estimate the size growth to be probably 2MBytes for everything -- sounds small for desktops/servers, not so for embedded devices. As such, it isn't something that should be added to an embedded router firmware, but certainly would be feasible via third-party Entware. But this thread is about a firmware that provides adblock support natively in the firmware, so.

    A reasonable trade-off would be to just allow the regex list you posted here, but permit A-Z and underscore to the list (A-Z because you can then remove -i flag on grep). Other characters would need to be allowed too, since that pipeline isn't just looking at the FQDN portion, it's looking at the entire line, which means allowing spaces and periods (you got these), as well as tabs. Stripping out carriage returns like you are is also smart. I think you may be able to do all of this in a single awk or sed command, rather than using lots of pipes (which require fork/exec for each one), but you'll need to see if Busybox awk's regex is extended or not (Busybox awk != gawk (GNU awk)), else consider Busybox sed (which does support extended syntax via -E flag). This is exactly what awk/sed are made for.

    In short: all it's going to take is one of those adblock blacklist servers to include one botched line or byte and the entire thing breaks. The above reports are proof. Folks using this feature are thus at the mercy of all of the servers/providers/content in all of the blacklist URLs. Tread lightly. :)
     
  97. Pasha_ZZZ

    Pasha_ZZZ Serious Server Member

    Code:
    sed 's/\r$//;/^[ \t]*#/d;/[\t ][[:alnum:]\._-]*$/!d'
    Tested on internal BusyBox sed, works for me.
     
    AndreDVJ and koitsu like this.
  98. monoton

    monoton Serious Server Member

    Updated to 2018.4 with NVRAM reset on three routers connected as AP's. Two Linksys EA6400 and one Linksys EA6700. All three configured with a guest network.
    Works fine over ethernet, but with wireless I get a message that says the password is wrong even though it's not (checked and retyped several times). Says the password is wrong for the guest WiFi as well. Tried to connect with a Laptop and two phones.
    Tried once more with NVRAM reset and reconfigured all three, same problem.
    WPA2 Personal - AES

    Downgraded back to 2018.3 and configured the same as before and I have no problem connecting the devices over WiFi.
     
  99. Aristide

    Aristide New Member Member

    Happened to me times ago with tomato shibby, was a browser issue, from then I always clean the cache before setting up new pw. btw maybe its not that case, im just giving my experience :)
     
  100. Tomato Mike

    Tomato Mike Network Newbie Member

    Sounds like you are experiencing an issue I've reported a few times already:

    "Also, the bug that I mentioned a few months back still stands, related to 'Enabling' the Guest Wifi (Virtual Wireless). If you enable Virtual Wireless on 2.4ghz, the radio will basically not work ever again, until you clear NVRAM. Nothing will legitimately connect to 2.4ghz (on regular or virtual wireless) until NVRAM is cleared. I learned that this bug still exists the hard way."

    I'm not sure which radios you enabled your guest WiFi on, but it seems you're having the same issue that I had. I get around this now by using the "Avoid performing an NVRAM commit" option by default. So, if I have to turn on my guest network for somebody, I just reboot the modem and everything goes back to the way it was (before the guest network was turned on), and everything continues to work fine.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice