Script: Adblock - not so lean

Discussion in 'Tomato Firmware' started by jerrm, Mar 13, 2016.

  1. AtTheAsylum

    AtTheAsylum LI Guru Member

    Thanks jerrm :) Any opinions on which is best?

    If pixelserv returns an error does that mean scripts can detect this and display a warning or attempt to display something else?

    BTW - love your adblock script :)
     
  2. AtTheAsylum

    AtTheAsylum LI Guru Member

    Another thing :)

    I figured out the reason for the error when saving. I checked the console and saw:

    line 330: not in a function

    I checked the source of adblockweb.sh and at line 301 found:

    local file=""

    I changed that to:

    file=""

    and the error went away. Seems strange? Surely others would have received this?
     
  3. jerrm

    jerrm Network Guru Member

    I have no interest in importing a ca cert into my clients, so I use pixelserv. For myself, the appeal of router based adblock is to not touch the client. I'd rather have the less than ideal output than muck with clients. To each his own.

    The "error" returned is aborting the tls certificate negotiation, to my knowledge that part of the process in the bowels of the browser and not directly exposed to javascript (or other script engine) via normal page script, it's just a failed connection. Maybe via one of the browser add-in interfaces.
     
    AtTheAsylum likes this.
  4. jerrm

    jerrm Network Guru Member

    This was discussed earlier and only impacts the most recent builds (shibby 138 similarly recent Toastman). The code in question started as a function and was moved inline. Earlier versions of busbox shell did not (but should have) thrown an error and it was never caught.

    I really need to fix the posted script, but I doubt I will ever move beyond 132 with Shibby, so it hasn't been a on my personal radar.
     
    AtTheAsylum likes this.
  5. srouquette

    srouquette Network Guru Member

    appreciative bump. Any plan to add the pixelsrv tls to the script?
     
  6. Bird333

    Bird333 Network Guru Member

    I think I need a new pixelserv. The gui is saying it's not responding. Which version works for ARM?
     
  7. dasfast

    dasfast Network Guru Member

    Pixelserv V35.HZ13 Works for my R6400 that is ARM.
     
  8. Bird333

    Bird333 Network Guru Member

    Why does the adblock webpage tell me that the configuration has changed when I haven't made any changes?
     
  9. rootMBX

    rootMBX Networkin' Nut Member

    Thank you for all your hard work, jerm. I've been a user of your awesome script since you first posted on the previous thread. The community appreciates you.

    Just wanted to post my config/tweaks to the process. Made config/param changes to write everything to /tmp/adblock. It saves writes to the Router's flash and RAM is so much faster than JFFS. List (re)generation and even browsing latency is improved.

    The following script installs entware (for https-capable wget), downloads the script (with my params), config, white/black lists, and pixelserv to /tmp/adblock and kicks off the process with a cron job.

    I throw it in the Administration -> Scripts -> Firewall section.

    Enjoy and thanks for making this easy to accomplish!

    NOTE: I'm pretty sure the version I have on pastebin is 2015-11-11. Not 100%. And I recommend you do this on a router with enough RAM. >=128MB is safe. 64MB might be enough too if you modify to use less lists.
     

    Attached Files:

  10. koitsu

    koitsu Network Guru Member

    Busybox wget is HTTPS-capable (it actually runs "openssl s_client" as a middle-man for the encryption/decryption), with better support for servers that use or require TLS SNI since Busybox 1.25.0 (I know because I'm the patch author (further reference/details)). So as long as you're using a recent TomatoUSB firmware, you shouldn't need Entware-ng for wget, unless there's a GNU wget feature/flag you're relying on (and if so, could I know what it is could I could implement it in Busybox wget?)
     
  11. Bird333

    Bird333 Network Guru Member

    Bump.
     
  12. rootMBX

    rootMBX Networkin' Nut Member

    Wrote the script long ago, before that patch was applied. I'll probably cut that out and update jerm's at some point. Thanks for the info!

    What's the word on the new built-in Adblocker in shibby's GUI or even AB Solution project (Merlin only) I just came across? Worth switching over to either?
     
  13. koitsu

    koitsu Network Guru Member

    Oops, sorry -- while dealing with unrelated stuff, I realised I got the version number wrong. It got introduced into official Busybox as of 1.26.0. Tomato MIPS uses 1.25.0 with my wget patch applied, so it works there (commit reference for Toastman, and I can confirm it does). ARM, on the other hand, does not have it (ARM uses Busybox 1.23.2 -- don't ask, I'm in a PM with Toastman about all that, there is chaos).
     
    rootMBX likes this.
  14. twentyninehairs

    twentyninehairs Reformed Router Member


    I used Shibby's ad block for a while, it does it's thing very well. The debugging is not as good as this UI, but I never had any big issues without a fancy debugger. When I had something not resolving, I simply assumed the adblocker was causing the name to not resolve, and added whatever dns name wasn't working to the whitelist. I never had any issues doing that. I can see a lot of situations that might really require a lot of debugging, but I never ran into a situation that needed it. It also doesn't have that fancy pixel server, but that wasn't any big thing.

    The truth is, that I just now moved away from Shibby builds on the R7000 to Toastman due to too many bugs with basic functionality. I very much agree with Toastman's philosophy, and his focus on core router functionality shows in his build for at least the R7000. From my experience, I think that maybe Shibby might work work well on MIPS hardware while Toastman works better on ARM. I'm not entirely sure that is always the case, but I do think that the method of blocking ads is much less important than most of the other features. Shibby's version of ad blocking worked very well, was very simple, and problem free. But I'm definitely not going to use it simply for that feature.
     
    rootMBX likes this.
  15. twentyninehairs

    twentyninehairs Reformed Router Member

    I will say that when moving from Shibby V138 to Toastman V9008.6 for the R7000, I was a little bummed about the lack of adblocking in Toastmans's build. Toastman had too many basic features fixed that were broken in Shibby's build for the R7000; such as bugs in VLAN configurations, dyndns local WAN ip address resolution (I think also wanup triggering), and nvram issues with backup/restore functions. This adblocking feature was the only thing I really missed from the Shibby build. So I googled for scripts... and found this!

    While this adblock system is a huge hack, I feel like it is an upgrade, not a downgrade from what I gave up in Shibby's build. Probably the larger the hack, the more difficult it is for the end user to maintain; but this was put together amazingly well. I'm more than happy with my Toastman setup and this adblock; I'm ecstatic.

    Thank you jerrm for this amazing system. I've never seen anything like it.
     
    rootMBX and visceralpsyche like this.
  16. ambiance

    ambiance Networkin' Nut Member

    Recently I've noticed a lot of garbled text at the top and it seems to happen while watching Netflix on my PS3. I'm using the latest version of your script on a flash drive on a R7000 using the latest version of Toastman.

    Here's a sample:

    recently resolved hosts:
    �eb 11 10:34:26 query[A] dns}asu[31743]:
    Veb 11 14:02:24 192.168.1.15 ichnaea.netflix.com
    F�b 11�10:34:01 art-2.nflximg.n�t 192�168.1.15
    F�b 11 1s:=3:22 query[A] d�soasq[14826]:
    Fmb 11 10:34:1x DHCPACK(br0) dnsmasq-dhcp[31743]:
    Fe� 11 17:01:10 qumry[A] dns}asq[19905]:
    Fe� 11 97:12:08 from appboot.netflix.com
    Fej 11 14:02:46 192.168.1n15Fmb tt�aka�.�flximg.com
    Fej 11 14:00:35 192.178.1.15 a�t-2.nflximg.net
    Fej 11 10:44:09 query[A] dlsmasq[31743]:
    Fef 11 16:24:�7 query[A] dnsmasq[29390]:
    Fed 11 16:34:59 192.168.1.4 54-149-26-14�-push.np.communication.playstation.net
    1.3" class="line" data-hostname="ichnama.netglix/com">Fec 11 14:00:22 192.168>1.3 ichnama.netglix/com
    Feb�11 17:01:27 api.steampow�red.com 192.168�1.10
    Feb011 17:02:08 secure.netflix.com 192.168.1.5
    Feb!11 1�:46:17 dnsmisq[31743]: apm-global.netflix.com query[A]
    8.1.3Feb" class="line" data-hostname="api-global.netflix.co�">Feb 51 14:11:40 192.1>8.1.3Feb api-global.netflix.co�
    Feb 1� 10:33:23 192.168.1.15 �rt-0.nflximg.net
    Feb 1q 17:00:15 192.168.1.10 api.stecmpoweret.com
    Feb 15 17:00:31 192.168.1.90 telk.google.com
    Feb 11�16:21:55 appboot.netflkx.com 192.168.1.3
    Feb 11`13:53:13 pollservmr.lastpass.com 192.16<.1.2
    Feb 11 q0:47:18 192.368.1.1� api=global.nut&lkx.com

    nslookup: can't resolve 'query[A]' BusyBox v1.23.2 (2017-01-21 01:37:51 ICT) multi-call binary. Usage: nslookup [HOST] [SERVER] Query the nameserver for the IP address of the given HOST optionally using a specified DNS server sh: syntax error: unexpected "(" nslookup: can't resolve 'qumry[A]' nslookup: can't resolve 'from' nslookup: can't resolve '192.168.1n15Fmb' nslookup: can't resolve '192.168' BusyBox v1.23.2 (2017-01-21 01:37:51 ICT) multi-call binary. Usage: nslookup [HOST] [SERVER] Query the nameserver for the IP address of the given HOST optionally using a specified DNS server BusyBox v1.23.2 (2017-01-21 01:37:51 ICT) multi-call binary. Usage: nslookup [HOST] [SERVER] Query the nameserver for the IP address of the given HOST optionally using a specified DNS server nslookup: can't resolve 'apm-global.netflix.com' nslookup: can't resolve '192.1' BusyBox v1.23.2 (2017-01-21 01:37:51 ICT) multi-call binary. Usage: nslookup [HOST] [SERVER] Query the nameserver for the IP address of the given HOST optionally using a specified DNS server BusyBox v1.23.2 (2017-01-21 01:37:51 ICT) multi-call binary. Usage: nslookup [HOST] [SERVER] Query the nameserver for the IP address of the given HOST optionally using a specified DNS server nslookup: can't resolve '192.368.1.1�'
     
  17. koitsu

    koitsu Network Guru Member

    @ambiance This looks like bad RAM, or kernel memory space that is being corrupted somehow. I can tell from the way it manifests, and how the corruption is happening. What isn't making sense is why there are things like actual HTML content in some of the logging lines. Tracking this down remotely is basically impossible.

    This doesn't appear to be a pixelserv problem. I can tell because some of the logging lines (which are sent through syslog, thus end up handled by syslogd, then written to /var/log -- unless you changed the logging path or destination, or are using a remote syslog server?) are for dnsmasq, which most certainly are not pixelserv.

    There's even evidence of the corruption being passed on to shell utilities like nslookup. I've underlined/bolded/italicised the character:

    Code:
    Query the nameserver for the IP address of the given HOST optionally using a specified DNS server nslookup: can't resolve '192.368.1.1�'
    
    This looks like a bit-level RAM error (ex. ASCII letter "1" is 0x31 (%00110001 in binary), while "3" is 0x33 (%00110011 in binary -- note the last bit)). Certainly it meant 192.168.1.1. Another example:

    Code:
    Fe� 11 97:12:08 from appboot.netflix.com
    
    The "b" in "Feb" is corrupted (can't tell what it becomes), but the timestamp should probably be 17:12:08; ASCII letter "1" is 0x31 (%00110001), while ASCII letter "9" is 0x39 (%00111001). So it's spread across multiple bits, i.e. isn't limited to just bit #3.

    If only vendors used ECC RAM... :(

    If you've overclocked this router in any way, stop immediately, though damage may have already been done/made permanent.
     
    Last edited: Feb 12, 2017
    ambiance and visceralpsyche like this.
  18. ambiance

    ambiance Networkin' Nut Member

    @koitsu It's never been overclocked and that is not good news if it's the hardware. I have IP, Bandwidth and dnsmasq being written to the flash drive the script is running on. Does the evidence suggest that the drive itself could be the problem? I've swapped it with fingers crossed.
     
  19. koitsu

    koitsu Network Guru Member

    As I said: "I can tell because some of the logging lines (which are sent through syslog, thus end up handled by syslogd, then written to /var/log -- unless you changed the logging path or destination, or are using a remote syslog server?) are for dnsmasq, which most certainly are not pixelserv."

    /var/log on Tomato is a directory that is a RAM-based filesystem (specifically, /var is a symlink to /tmp/var, and /tmp is tmpfs, which is pure RAM). The default logfile is /var/log/messages. syslogd (the daemon) is what writes to this file. Programs send logging lines to syslogd using the syslog() library call.

    If you didn't change the logging destination, then the issue has nothing to do with a flash drive. If you're using a remote syslog server, then the issue has nothing to do with a flash drive.
     
    ambiance likes this.
  20. ambiance

    ambiance Networkin' Nut Member

    I'm not using a server or flash drive for syslog and if I understand this correctly it's almost certainly not the drive as the programs being called are located in RAM which may have one, two, three (ha, ha, ha) bad sectors. How would I go about diagnosing the memory? If it's severely complicated or not possible via software I'm SoL.

    I'd be much happier if this happened to my N66U and not the R7000. FWIW, there hasn't been any corruption since changing flash drives and I wasn't able to find any bad sectors on what I thought may be the culprit. I apologize for making you repeat yourself as my comprehension level on these matters requires me to be told at least twice or thrice depending on the day.

    Thanks for your input
     
  21. koitsu

    koitsu Network Guru Member

    The binaries/programs themselves are part of the flash (NAND), not part of RAM.

    Tomato partitions the flash and uses read-only (ROM-based) filesystems. Those flash partitions are mounted as Linux filesystems (specifically squashfs). Do a mount sometime to see what I'm talking about.

    If the binaries/programs themselves on those partitions/filesystems were broken -- e.g. somehow the bad NAND block support got bypassed for the flash, and "bad data" was written to the flash -- then I would not expect this behaviour to be happening for *multiple* binaries. It would more likely be isolated to a single one, and the way it would manifest would not be "corrupted logs", it would be that the binary segfaulted (crashed) or wouldn't run at all (many symptoms are possible, too many to list here).

    For a program to work, it needs RAM (the kernel allocates this on the fly, etc.). I won't get into details about how ELF binaries work or ld.so loaders, etc...

    The R7000 has a 256MB RAM chip somewhere on it (or possibly a pair of 128MB). I can't find good teardown PCB photos with the HSF removed, so I can't show you this. But the point is: RAM is separate/unrelated from NAND flash.

    What you're showing looks like corrupted logging data, being written to /var/log/messages (which is stored on tmpfs, i.e. a RAM-based filesystem (think "ramdisk")). What's more important is that binaries being used, such as nslookup, are being fed/given "bad/corrupted data" (ex. nslookup 192.368.1.1). This is almost certainly caused by bad RAM, i.e. what's being stored in RAM is "192.168.1.1", then when nslookup runs and reads the data back, it gets 192.368.1.1.

    The router should be behaving EXTREMELY oddly in general given this behaviour, but it also depends partially on where the bad RAM is on the chip itself. 256MB is a lot -- there may just be a small region (say 512 bytes?) of the RAM that has a problem. Given how kernels allocate memory, the problem would only show up once a program uses that region of RAM that's bad.

    This is why I said it's too bad vendors don't use ECC RAM.

    You can't diagnose the memory. The only option is to RMA or replace the router.

    Edit: It looks like Entware-ng has a package called memtester that can test some (not all -- it can't touch kernel memory!) of the RAM. However, it's not a "user-friendly" program in comparison to, say, memtest86. Here's why:

    memtester uses standard malloc() allocation for memory. What this means is two things: 1) libc and the kernel end up picking whatever area of memory that provides that amount (e.g. if you ask for 100MB, it'll give you 100MB, but it might not be where the bad RAM would be), 2) it can (and will!) use swap (e.g. if you ask for 512MB and your router has only 256MB RAM but you have swap enabled/set up, it'll test some portion of swap as well). Swap I/O on Tomato is EXTREMELY tedious -- it's literally the slowest thing I've ever seen (due to USB). It's painful, and you want to avoid hitting swap for a test like this if possible.

    It also (very understandbly) cannot test memory regions that are already allocated by existing programs. It's literally "just a program that allocates memory then tests what its given". It'll also suck up all the CPU time on the router (it has to), so uh, yeah, using your router for something while this is going on = probably going to be slow.

    Finally, the likelihood of this program making your router useless (i.e. you will need to reboot it after) is EXTREMELY high. This is because picking the wrong memory size can (WILL!!!) result in Linux OOM killer kicking in, which can start to kill off things like dnsmasq, httpd, syslogd, EVERYTHING! I can tell you right now this is the truth -- I've confirmed it.

    In other words: I cannot advise use of memtester for just a "simple program to run to test stuff". For example, my router had 240MB of RAM free, yet memtester 200M resulted in Linux OOM kicking in and memtester being killed by the kernel immediately. You really have to understand the Linux VM and kernel bits to understand how/why all this works.

    If you do try it, well, it isn't definitive because of all of the above. But here's a brief example sample run (I aborted it mid-way):

    Code:
    root@gw:/tmp/home/root# memtester 100M
    memtester version 4.3.0 (32-bit)
    Copyright (C) 2001-2012 Charles Cazabon.
    Licensed under the GNU General Public License version 2 (only).
    
    pagesize is 4096
    pagesizemask is 0xfffff000
    want 100MB (104857600 bytes)
    got  100MB (104857600 bytes), trying mlock ...locked.
    Loop 1:
      Stuck Address       : ok
      Random Value        : ok
      Compare XOR         : ok
      Compare SUB         : ok
      Compare MUL         : ok
      Compare DIV         : ok
      Compare OR          : ok
      Compare AND         : ok
      Sequential Increment: ok
      Solid Bits          : setting  42
    root@gw:/tmp/home/root#
    
    Now to go reboot my router because I've induced the Linux OOM killer more times than I care to trust. :)

    Edit: large edit talking about memtester from Entware-ng.
     
    Last edited: Feb 13, 2017
    ambiance likes this.
  22. ambiance

    ambiance Networkin' Nut Member

    The warranty has long since expired leaving me the options of replacing it or a good thrashing. Everything seems to be working fine and I hope it can keep that up.

    Edit: So in order for it to work properly it would need full access to memory which is essentially impossible. Perhaps it could be programmed to move all programs in reserved memory when all free has been checked, but that would be beyond my abilities assuming it is even possible. If only Netgear had a diagnostics tool...
     
    Last edited: Feb 13, 2017
  23. QSxx

    QSxx Network Guru Member

    Just a quick update (?) of source lists I currently use - If someone has others to add for more comprehensive list, please do:

    Code:
    ## MVPS HOSTS (~492 KB) [default]:
    SOURCES="$SOURCES http://winhelp2002.mvps.org/hosts.txt"
    
    ## pgl.yoyo.org (~61 KB) [default]:
    SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"
    
    ## AdAway mobile ads (~14 KB):
    SOURCES="$SOURCES https://adaway.org/hosts.txt"
    
    ## The Cameleon Project (~640 KB):
    SOURCES="$SOURCES http://sysctl.org/cameleon/hosts"
    
    ## hpHosts ad/tracking servers (~1,75 MB):
    SOURCES="$SOURCES http://hosts-file.net/ad_servers.txt"
    
    ## hpHosts ad/tracking/malicious servers (~6M! replaces hpHosts ad/tracking list):
    #SOURCES="$SOURCES http://hosts-file.net/download/hosts.txt"
    
    ## hpHosts ad/tracking/malicious servers appendix (~980 KB replaces hpHosts ad/tracking list):
    SOURCES="$SOURCES http://hosts-file.net/hphosts-partial.txt"
    
    ## MalwareDomainList.com (~37 KB):
    SOURCES="$SOURCES http://www.malwaredomainlist.com/hostslist/hosts.txt"
    
    ## Hosts File Project (~2,77 MB) beware this list - breaks many things:
    SOURCES="$SOURCES http://hostsfile.mine.nu/Hosts"
    At the time of posting all links were functional (checked) with no errors and their sizes were updated accordingly (so you have an idea how "heavy" each one is)

    !!!NOTE!!! - Last list breaks whole lot of things including-but-not-limited-to :) google sync/chrome sync, samsung phone internet over wifi check, etc... (unless whitelisted ofcourse) - so be extra careful. If something breaks, it's better to disable that one source and rebuild blocklist or try again and then go hunting for host that got blocked...
     
    Last edited: Mar 9, 2017
  24. koitsu

    koitsu Network Guru Member

    @QSxx http://adaway.org/hosts.txt needs to be https://adaway.org/hosts.txt -- the HTTP version redirects to the HTTPS version (see for yourself: it returns an HTTP 301 with a Location: header of https://adaway.org/hosts.txt), and is hosted via CloudFlare now, which requires that the fetching client support TLS SNI (see previous posts in this thread about wget and that behaviour, or search the forum for "TLS SNI wget" from me, you'll see I'm the author of the implementation in Busybox's wget). There's zero guarantee the HTTP redirection will remain in place indefinitely, so best update the URL now.
     
    Elfew and QSxx like this.
  25. QSxx

    QSxx Network Guru Member

    Will do - and thank you :) - previous post will be corrected
     
  26. Rangaistus

    Rangaistus Reformed Router Member

  27. ofcoursemyhorse

    ofcoursemyhorse Reformed Router Member

    I'm using Steven Black's host file and it's using 0.0.0.0 instead of 127.0.0.1. Any disadvantage using 0.0.0.0 on K26RT-N? And why is adblock showing 24435 domains when downloaded hosts file contains 35,566 unique domains?

    Thanks!
     
  28. rootMBX

    rootMBX Networkin' Nut Member

    This is exactly what I ended up doing for my RT-AC3200 and I never looked back. There are strange issues with choosing channels on the two 5 GHz radios that are not available for choosing on the Asus/Asus-Merlin firmwares, but as long as I stay within those channels, everything works well. (Seems like a general Tomato issue, not just Toastman)
     
  29. Wolfer

    Wolfer Network Newbie Member

    Netgear AC1450 Tomato Firmware 1.28.0000 -129 K26ARM USB AIO-64K (R6300v2)

    I really like the jerm version of AdBlock (thx, have be using it for quite some time successfully) and prefer it over the new embedded AdBlock in the Shibby updates especially because of the AdBlock home page which gives a live look at the blocked/resolved hosts. The only problem is that the inability to load https lists is starting to limit the number of lists that will load (including lists that are http addresses, but redirect to https like Adaway)

    I tried to resolve the issue by using rootMBX's code to install the entware wget for https-capable wget
    When I ran it, it returned the following errors:

    Connecting to qnapware.zyxmon.org (81.4.123.217:80)
    wget: can't open 'entware_install_arm.sh': File exists
    Info: Checking for prerequisites and creating folders...
    Warning: Folder /opt exists!
    Warning: Folder /opt/bin exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/etc exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/include exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/lib exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/sbin exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/share exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/tmp exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/usr exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Warning: Folder /opt/var exists!
    Warning: If something goes wrong please clean /opt folder and try again.
    Info: Opkg package manager deployment...
    Connecting to qnapware.zyxmon.org (81.4.123.217:80)
    wget: can't open 'opkg': File exists
    Connecting to qnapware.zyxmon.org (81.4.123.217:80)
    wget: can't open 'opkg.conf': File exists
    Connecting to qnapware.zyxmon.org (81.4.123.217:80)
    wget: can't open 'ld-2.20.so': File exists
    ln: ld-linux.so.3: File exists
    Connecting to qnapware.zyxmon.org (81.4.123.217:80)
    wget: can't open 'libc-2.20.so': File exists
    ln: libc.so.6: File exists
    Info: Basic packages installation...
    /opt/bin/opkg: line 1: syntax error: unexpected "("
    /opt/bin/opkg: line 1: syntax error: unexpected "("
    Connecting to qnapware.zyxmon.org (81.4.123.217:80)
    wget: can't open '/opt/usr/lib/locale/locale-archive': No such file or directory
    Info: Congratulations!
    Info: If there are no errors above then Entware.arm successfully initialized.
    Info: Add /opt/bin & /opt/sbin to your PATH variable
    Info: Add '/opt/etc/init.d/rc.unslung start' to startup script for Entware.arm services to start
    Info: Found a Bug? Please report at https://github.com/zyxmon/entware-arm/issues
    /tmp/.wxZvtlQS: line 39: /opt/etc/init.d/rc.unslung: not found
    /opt/bin/opkg: line 1: syntax error: unexpected "("


    I was hoping you might be able to shed some light on why the install is failing

    As an aside, I tried upgrading to Shibby 138 and 140, since the https capability is built in, but I started getting the following recurring error (~every 1 minute) in the logs "unknown user.debug preinit(1): dnsmasq terminated unexpectedly, restarting." which obviously made it impossible to maintain a stable connection. Wondering if anyone else had the same experience and whether you were able to resolve the issue.

    Thanks



     
  30. Frequenzy

    Frequenzy Networkin' Nut Member

    no need to do any of that, just edit the main adblock.sh and find wget_opts and add "--no-check-certificate"
     
  31. Wolfer

    Wolfer Network Newbie Member

    Thanks
    I gave that a whirl, but it is returning an error as you can see below

    wget: unrecognized option `--no-check-certificate'
    BusyBox v1.21.1 (2015-04-23 11:31:04 CEST) multi-call binary.

    Usage: wget [-csq] [-O FILE] [-Y on/off] [-P DIR] [-U AGENT] [-T SEC] URL...

    Retrieve files via HTTP or FTP

    -s Spider mode - only check file existence
    -c Continue retrieval of aborted transfer
    -q Quiet
    -P DIR Save to DIR (default .)
    -T SEC Network read timeout is SEC seconds
    -O FILE Save to FILE ('-' for stdout)
    -U STR Use STR for User-Agent header
    -Y Use proxy ('on' or 'off')

    ADBLOCK[11819]: Unchanged: http://winhelp2002.mvps.org/hosts.txt (Last-Modified: Fri, 15 Sep 2017 14:51:32 GMT)
    wget: unrecognized option `--no-check-certificate'
    BusyBox v1.21.1 (2015-04-23 11:31:04 CEST) multi-call binary.

    Usage: wget [-csq] [-O FILE] [-Y on/off] [-P DIR] [-U AGENT] [-T SEC] URL...

    Retrieve files via HTTP or FTP

    -s Spider mode - only check file existence
    -c Continue retrieval of aborted transfer
    -q Quiet
    -P DIR Save to DIR (default .)
    -T SEC Network read timeout is SEC seconds
    -O FILE Save to FILE ('-' for stdout)
    -U STR Use STR for User-Agent header
    -Y Use proxy ('on' or 'off')

    ADBLOCK[11819]: Failed: http://adaway.org/hosts.txt
    ADBLOCK[11819]: Downloading: http://hosts-file.net/ad_servers.asp
    ADBLOCK[11819]: Failed: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
    wget: unrecognized option `--no-check-certificate'
    BusyBox v1.21.1 (2015-04-23 11:31:04 CEST) multi-call binary.

    Usage: wget [-csq] [-O FILE] [-Y on/off] [-P DIR] [-U AGENT] [-T SEC] URL...

    Retrieve files via HTTP or FTP

    -s Spider mode - only check file existence
    -c Continue retrieval of aborted transfer
    -q Quiet
    -P DIR Save to DIR (default .)
    -T SEC Network read timeout is SEC seconds
    -O FILE Save to FILE ('-' for stdout)
    -U STR Use STR for User-Agent header
    -Y Use proxy ('on' or 'off')
     
  32. Frequenzy

    Frequenzy Networkin' Nut Member

    it looks like you are running an old tomato version
     
  33. Wolfer

    Wolfer Network Newbie Member

    Yes, I mentioned in my original post that I had tried upgrading to Shibby 138 and 140, but was getting dnsmasq errors (requoted below). Any insight into what might be causing the issue? The only thing I found on the issue was a possible race between AdBlock and dnsmasq, but wasn't sure since I don't have any special dnsmasq inputs in Dnsmasq custom configuration (I do it in the Whitelist configuration). And of course still wondering if there is any way to resolve the https issue in the version of Tomato that I am running? Thanks.

    Requoted from earlier post:
    As an aside, I tried upgrading to Shibby 138 and 140, since the https capability is built in, but I started getting the following recurring error (~every 1 minute) in the logs "unknown user.debug preinit(1): dnsmasq terminated unexpectedly, restarting." which obviously made it impossible to maintain a stable connection. Wondering if anyone else had the same experience and whether you were able to resolve the issue.
     
  34. rs232

    rs232 Network Guru Member

    I'm experiencing three issues with the script, hopefully nothing serious.

    Background: I'm trying to resolve an issue with my Samsung Smart TV as after power on when I try to run the SmartHub it says: SmartHub is being updated and I need to wait around 30 seconds, every single time I power on the TV, to be able to run SmartHub. I thought this to be an issue with the TV itself but after reading up in Internet somebody suggested that router blocking/filtering might be the issue. It suddenly came to my mind that adblock is running on my router so I went playing with the settings.

    First of all I have enabled logging and I can see that there are 3 hosts indeed (Samsung related) filtered out by adblock:
    ads.samsungads.com
    config.samsungads.com
    log-config.samsungacr.com

    So I have added them to the whitelist. Here the issues:

    1) if I click the [+w] next to the host in the log the host does get added to the whitelist indeed but the following pop up appears:

    ScreenShot061.png

    2) The host appears to be added at the bottom of the whitelist but after an adblock fresh start I still see
    the hosts being blocked (and logged). This happend whether I add it manually or via the GUI with the [+w]

    3) When runningfrom the shell I get:
    Code:
    root@tomato36k:/tmp/mnt/usb/adblock# ./adblock.sh update
    ADBLOCK[10105]: Running as /mnt/usb/adblock/adblock.sh update
    ADBLOCK[10105]: Using config file /mnt/usb/adblock/adblock.ini
    ADBLOCK[10105]: Ignoring extra config file /cifs1/adblock/adblock.ini
    ADBLOCK[10105]: Requested list mode is OPTIMIZE
    ADBLOCK[10105]: Enabling dnsmasq logging
    ADBLOCK[10105]: Logging previously enabled
    ADBLOCK[10105]: Logging to syslog
    ./adblock.sh: local: line 1056: not in a function
    ADBLOCK[10105]: Creating web link /www/user/adblock.sh
    ADBLOCK[10105]: Web interface should be available at http://10.10.10.1/user/adblock.sh
    ADBLOCK[10105]: Adding tomato menu item
    ADBLOCK[10105]: Blocklist active in OPTIMIZE mode
    ADBLOCK[10105]: Config or script has changed - rebuilding list
    ADBLOCK[10105]: Download starting
    ADBLOCK[10105]: Unchanged: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext (Last-Modified: Fri, 24 Nov 2017 11:53:05 GMT)
    ADBLOCK[10105]: Unchanged: http://winhelp2002.mvps.org/hosts.txt (Last-Modified: Fri, 15 Sep 2017 14:51:32 GMT)
    ADBLOCK[10105]: Filters unchanged
    ADBLOCK[10105]: Blocklist does not exist
    ADBLOCK[10105]: Generating /var/lib/adblock/blocklist - OPTIMIZE mode
    ADBLOCK[10105]: Blocklist generated - 4 seconds
    ADBLOCK[10105]: 9725 unique hosts to block
    ADBLOCK[10105]: Setting up 10.10.10.254 netmask 255.255.255.0 on br0:adblk
    ADBLOCK[10105]: Setting up pixelserv on 10.10.10.254
    ADBLOCK[10105]: pixelserv[10440]: /mnt/usb/adblock/pixelserv version: V35.HZ13 compiled: Oct 31 2015 18:04:47 options: 10.10.10.254
    ADBLOCK[10105]: Writing File /etc/dnsmasq.custom
    ADBLOCK[10105]: CONF file /etc/dnsmasq.custom unchanged
    ADBLOCK[10105]: Restarting dnsmasq
    ADBLOCK[10105]: ...
    ADBLOCK[10105]: Done.
    ADBLOCK[10105]: Exiting /mnt/usb/adblock/adblock.sh 0
    The:
    ./adblock.sh: local: line 1056: not in a function
    appears to be new to me.

    I am running Kille72's 2017.3

    I'm not sure if this is a little bug, a peculiarity of the hosts I'm adding or perhaps something else I'm missing here.

    Thanks
     
  35. damionhh

    damionhh Networkin' Nut Member

    i dont know if this thread is sitll alive but i have been using this script for years and all has been working well. i particularly like the status view of what domains are blocked so i can troubleshoot problems. just recently i read an article on krebs about .cm domains and he suggested just blocking ALL of them. so i dutifully went to the Adblock Edit Lists option and added .cm to the blacklist. upon saving it said "ERROR List NOT Saved" even though when i SSHd into my router the blacklist file did indeed hold ".cm". then i tried NsLookup on "facebook.cm" and it returned a proper IP address. so i deactivated this script and tried the built in shibby adblock and it correctly blocks .cm domains if i enter them in the blacklist box by returning 0.0.0.0. any ideas what the problem is? i would prefer to use this script since the status screen is robust but if the blacklist doesnt work...

    Tomato Firmware 1.28.0000 MIPSR2-140 K26AC USB AIO-64K
    Latest AdBlock script installed via instructions using wget and sh.
     
  36. HunterZ

    HunterZ Network Guru Member

    Just updated my RT-N66U to TomatoFresh, and my adblock setup isn't working right.

    First it complained that it couldn't create a /etc/dnsmasq/hosts/zzz.adblock.hosts, and it turned out to be because something else had created a *file* named /etc/dnsmasq/hosts. I renamed that and made a directory named hosts, and got past that error, but the web status still shows "blocklist: down" and an empty "recently blocked hosts" pane.

    According to the dnsmasq log, it's only reading /etc/resolv.dnsmasq, /etc/hosts (2 addresses), /etc/dnsmasq/dhcp-hosts (0 addresses), and hosts.old (my renamed hosts file; 5 addreesses). It does NOT mention /etc/dnsmasq/hosts/zzz.adblock.hosts at all.

    Any ideas?

    Edit: Figured it out:
    • Added this line to adblock.ini: hostlink=/etc/dnsmasq/hosts.adblock
    • Restored the router's configuration of having an /etc/dnsmasq/hosts *file* instead of a directory.
     
  37. JoeDirte

    JoeDirte Networkin' Nut Member

    FYI - @HunterZ - FreshTomato has Adblock built-in under Advanced. I was using the script from this thread until I moved off Toastman's build to FreshTomato recently.
     
  38. HunterZ

    HunterZ Network Guru Member

    Yeah, I saw that. Wasn't sure at first glance how to make it work with pixelserv, so I wanted to stick with my current setup for now.
     
    JoeDirte likes this.
  39. Beast

    Beast Network Guru Member

    I also switched to FreshTomato on my Asus RT-N16. Did you figure out how to use the built in adblock with pixelserv ?? If so, how?

    In your post above you said to add "hostlink=/etc/dnsmasq/hosts.adblock" to the .ini file. Was this done to get the built in adblock to work, or the "orginal" ? I still use the "orginal" script, which is on a usb thumbdrive along with your pixelserv. Using firmware: FreshTomato Firmware 2018.3.081 MIPSR2-beta K26 USB Mega-VPN

    The orginal setup is working fine, have made no changes to any of it.
    But if the built in version works as well as the orginal, i would like to give it a try.
     
  40. HunterZ

    HunterZ Network Guru Member

    It was to get the original script working. I'm running from a thumb drive in my RT-N66U.
     
  41. Beast

    Beast Network Guru Member

    So your doing the same thing I am. Do you know if the built in version can be made to work with pixelsev?

    FreshTomato Firmware 2018.3.081 MIPSR2-beta K26 USB Mega-VPN on an Asus RT-N16.
     
  42. HunterZ

    HunterZ Network Guru Member

    I have no idea. I figured it was going to be less effort to get my existing setup working. The built-in stuff doesn't have any special advantages as far as I could tell - in fact, I think it caches the lists in RAM which means they have to be re-downloaded and rebuilt every time you reboot.
     
  43. plikmuny

    plikmuny New Member Member

    Hi is there a way to make this script work automatically after reboot. i am having troubles with Adblocking once i reboot.
    Can you give another script so that even after i reboot, it works . Thanks
     
  44. HunterZ

    HunterZ Network Guru Member

    I configure my router to run it as a wanup script.
     
  45. plikmuny

    plikmuny New Member Member

    Is there a customised script to make it work with installation Folder being USB drive ?
    i am just wondering if it works flawlessly inside RAM, it should be possible to run it on USB with little Code optimisation right ? If yes, can you share such a customised srcipt for me pls....
    Thanks in Advance.
     
  46. AndreDVJ

    AndreDVJ LI Guru Member

    Actually the installation instructions in the OP caters for your requirement.

    It should ideally run from /opt/bin, but you can modify this path to anything you like.
     
  47. plikmuny

    plikmuny New Member Member

    When i change the path from /opt/bin to external USB with /mnt/jffs/adblock it says read only mode and it fails to create and execute the commands. What am i doing wrong here ?

    Eventhough i have set the path, it still installs in /opt/bin.... i dont know ho i change the path. here is the log of it.

    Connecting to goo.gl (172.217.22.110:80)
    Connecting to tomato-adblock.weebly.com (199.34.228.54:80)
    - 100% |*******************************| 2475 0:00:00 ETA

    adblock-install: PREFIX not set, looking for default folders
    adblock-install: installing binaries and scripts to /opt/bin, config to /opt/etc/adblock.ini
    Connecting to tomato-adblock.weebly.com (199.34.228.53:80)
    adblock-install.tgz 53% |**************** | 180k 0:00:00 ETA
    adblock-install.tgz 100% |*******************************| 335k 0:00:00 ETA

    adblock.sh
    adblockweb.sh
    adblock.changelog
    adblock.ini.default
    adblock.ini.readme
    pixelserv/arm/
    pixelserv/arm/LICENSE
    pixelserv/arm/README.md
    pixelserv/arm/pixelserv.tomatoware.performance.dynamic
    pixelserv/arm/pixelserv.tomatoware.performance.static
    pixelserv/arm/VERSION
    pixelserv/mips/
    pixelserv/mips/LICENSE
    pixelserv/mips/README.md
    pixelserv/mips/pixelserv.tomatoware.performance.dynamic
    pixelserv/mips/pixelserv.tomatoware.performance.static
    pixelserv/mips/VERSION
    pixelserv/mipsK24/
    pixelserv/mipsK24/LICENSE
    pixelserv/mipsK24/README.md
    pixelserv/mipsK24/pixelserv.mips.performance.dynamic
    pixelserv/mipsK24/pixelserv.mips.performance.static
    adblock-install: installing /opt/bin/adblock.changelog
    adblock-install: installing /opt/bin/adblock.ini.readme
    adblock-install: installing /opt/bin/adblock.ini.default
    adblock-install: installing /opt/bin/adblock.sh
    adblock-install: installing /opt/bin/adblockweb.sh
    adblock-install: installing /opt/bin/pixelserv.tomatoware.performance.static
    adblock-install: creating 'pixelserv' link for /opt/bin/pixelserv.tomatoware.performance.static
    adblock-install: a config file appears to exist - skipping config install


    can zou help me pls...
     
  48. AndreDVJ

    AndreDVJ LI Guru Member

    Have you followed this instruction?
    Code:
    # For a custom location uncomment and edit PREFIX value
    # export PREFIX=/opt/bin
    wget -O - http://goo.gl/GfA7cQ | sh
    It says to uncomment (i.e. take off that #) and edit PREFIX value (i.e. replace /opt/bin with something else)
     
  49. plikmuny

    plikmuny New Member Member

    Thanks a lot. it did the Trick. now it installed without any problem. Will play with hosts list and get it customised according to my needs.
     
  50. plikmuny

    plikmuny New Member Member

    Can anyone explain me is this adblocking method is better and more effective as the one by haarp on tomato or is there a method better than these 2 ?
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice