Script: Clean, Lean and Mean Adblocking

Discussion in 'Tomato Firmware' started by haarp, Apr 23, 2013.

  1. Michael Malone

    Michael Malone Network Newbie Member

    I couldn't care less about the sorting, just wish the whitelist would actually remove the entries from the blocklist without having to paste them space separated within the quotations in adblock.ini.default. I would like them pulled from a remote URL, but they are never removed from the blocklist.
     
  2. jerrm

    jerrm Network Guru Member

    Whitelists work. There must be something wrong with the download, or it is not there at the right time. Are you stripping carriage returns?
     
  3. Michael Malone

    Michael Malone Network Newbie Member

    Yes, I have been stripping the carriage returns. I use notepad++ and Textpad.
    I have tried it so many ways, I've lost count. I have gone back to just adding it to the end of the config and that isn't working either. I have formatted the list both as just the domain and and as 0.0.0.0 domain.com. None of them are whitelisted and from config, also do not appear in the GUI. I don't know if they are supposed to?

    ## Blacklist additional sites
    ## Remote Url added to [adblock.sh]
    ## (add hostnames inside the quotes, space-separated, without http://)
    BLACKLIST=""

    ## Whitelist sites from blocking
    ## Remote Url added to [adblock.sh]
    ## (add hostnames inside the quotes, space-separated, without http://)
    WHITELIST=""

    ### Blacklist and Whitelist (optional) ###
    ## Create the files "blacklist" and "whitelist" with your hosts, one per line
    ## Useful if you have many hosts in these (they generate faster as well!)
    wget http://pastebin.com/raw.php?i=XXXXXXXX -O - > "$prefix/whitelist"
    wget http://pastebin.com/raw.php?i=XXXXXXXX -O - > "$prefix/blacklist"

    I use Bulk-DNS-Lookup and all domains remain blocked, after ipconfig /flushdns

    "xxxx.com","192.168.1.254"
    "xxxxxxxxxxxx.com","192.168.1.254"
    "xxxxxxxxxx.com","192.168.1.254"
     
  4. jerrm

    jerrm Network Guru Member

    That is meaningless once copied into pastebin.
     
  5. Michael Malone

    Michael Malone Network Newbie Member

    I have also trying adding the lists before running adblock.sh and I have placed a "#" at the end of the blacklist.
    They blacklist seems to be working, I only had one domain in it. ... but the whitelist is still having a problem. Both lists appear properly in the GUI.

    I came up with 64 domains currently blocked with all my hosts combined, wanting to test things. I added them all to the whitelist. I test the same 64 domains after everything is up and running and only 3 have been whitelisted. Sometimes none, or only two.

    wget http://pastebin.com/raw.php?i=XXXXXXXX -O - > /tmp/adblock/whitelist && echo success!
    wget http://pastebin.com/raw.php?i=XXXXXXXX -O - > /tmp/adblock/blacklist && echo success!
    $PREFIX/adblock.sh
     
  6. jerrm

    jerrm Network Guru Member

  7. Michael Malone

    Michael Malone Network Newbie Member

    ok jerrm, I will try it ...

    Just to be sure, removed all the stuff I had tried, leaving all my sources enabled, of course. I pasted the same 64 domains that I know are blocked into the WHITELIST="" to see if there was any issue. They don't appear in the GUI, but I am guess that is normal.

    Anyway, 61/64 end up whitelisted and 3 remain blocked. WTH?
    It's not that I actually need or require them, just what I discover.

    I am using this utility to bulk process the domain list. Just grab a bunch of domains from one of the sources to test the domain whitelisting.

    Bulk DNS Lookup in Windows Powershell – Better than NSLookup!
    http://www.geekynick.co.uk/bulk-dns-lookup-in-windows-powershell-better-than-nslookup-2

    "s0.2mdn.net","192.168.1.254"
    "ssl.google-analytics.com","192.168.1.254"
    "www.geoplugin.net","192.168.1.254"
     
  8. HunterZ

    HunterZ Network Guru Member

    Haven't updated my adblock script in a while because it was running well for me, but I'm looking at the 2015-11-11 release because I'm moving adblock/entware/tomatoware from CIFS to USB.

    Looking at the new adblock.ini config system, I'm confused about a couple of things:
    1. Are the blacklist/whitelist implemented as config file settings, or as separate files, or either?
    2. I would suggest combining adblock.ini.readme and adblock.ini.default into a single example file, possibly commenting out everything other than the default sources setting. As they currently stand, it's weird that each of them has a subset of options (no sources in the former and pretty much nothing but sources in the latter).
     
    visceralpsyche likes this.
  9. Michael Malone

    Michael Malone Network Newbie Member

    Alright, I ran this with the same 64 domains in the whitelist and this has worked just as well as adding the domains to whitelist="" so that is a success. It didn't work with the blacklist, the domain ended up not blacklisted. I had to add the "#"

    wget http://pastebin.com/raw.php?i=XXXXXXXX -O - | tr -d '\r' > /tmp/adblock/whitelist && echo success!
    $PREFIX/adblock.sh

    However, these 3/64 domains remain blocked. why, why and why does this happen?
    I can only guess due to being contained in multiple sources, but that should have already been sorted out with the script.

    "s0.2mdn.net","192.168.1.254"
    "ssl.google-analytics.com","192.168.1.254"
    "www.geoplugin.net","192.168.1.254"
     
    Last edited: Mar 12, 2016
  10. jerrm

    jerrm Network Guru Member

    1: either or both
    2: maybe if I ever get around to cleaning docs up some more
     
    visceralpsyche likes this.
  11. HunterZ

    HunterZ Network Guru Member

    The weebly site uses a .zip file to distribute adblock, but .zip doesn't store Linux execute permissions. Using .tar.gz may be a better option.
     
  12. jerrm

    jerrm Network Guru Member

    More correctly, busybox unzip doesn't restore the permissions.

    Doubt I will change it. The install archive is a tarball. Having the zip format available is more windows friendly and simpler for the parameter-phobic.
     
  13. aksaraff

    aksaraff New Member Member

    After a couple of tries, I managed to get this working on my Linksys WRT54GS v.4 using help from this thread and the pixelserv one.

    When disabling pixelserv (by setting PIXEL_IP="0" in config), I see the blocklist working correctly as expected and no ads show in my browser windows (I tested this with multiple installations to make sure that it wasn't the AdBlock Plus extension doing its job).

    When enabling pixelserv (PIXEL_IP="254") and restarting adblock, I notice a strange behavior upon browsing to sites with embedded ads. The DNS poisoning correctly filters the ad and tracking requests to the correct IP (in this case, at 254) but somehow, this is re-directed to my router's homepage (at 1) so I get a bunch of login prompts (for my router's configuration page). A single page will result in multiple login requests which I have to cancel individually - not an ideal situation. My understanding was that pixelserv runs a simple server on the given IP and serves up a 1x1 gif image for all requests - but that's not what is happening here.

    I am attaching ifconfig and iptables output in case that helps in figuring out things. uYZZwhZ0 on paste bin
     
  14. HunterZ

    HunterZ Network Guru Member

    @aksaraff: What version of adblock are you trying to use? That iptables setup looks a lot different from the one I get using the newest adblock script, which sets up an adblk.fw chain. Also, it listens on an interface named br0:adblk instead of br0:1.

    Proper link to your pastebin: http://pastebin.com/uYZZwhZ0
     
  15. aksaraff

    aksaraff New Member Member

    The comments have the following entry -
    Code:
    ## Clean, Lean and Mean Adblock v4.5 by haarp
    
    The script was downloaded from the original post on this thread.
     
  16. WaLLy3K

    WaLLy3K Networkin' Nut Member

    The first post of the thread? Then you would likely want to try jerrms version.
     
  17. HunterZ

    HunterZ Network Guru Member

    I really wish we could get a moderator/admin to split the thread and archive everything before jerrm's version post.
     
    pharma, JoeyJoeJoe and WaLLy3K like this.
  18. jerrm

    jerrm Network Guru Member

  19. Michael Malone

    Michael Malone Network Newbie Member

    I am now using the slight mod below and no longer need to add "#" to the end of my pastebin.com file.
    The 3 domains which I previously discovered will not whitelist (using remove Url's) remain unexplained.
    s0.2mdn.net ssl.google-analytics.com and www.geoplugin.net

    # Download Whitelist
    wget http://pastebin.com/raw.php?i=XXXXXXXX -O - | tr -d '\r' > /tmp/adblock/whitelist && echo "" >> /tmp/adblock/whitelist && echo success!

    # Download Blacklist
    wget http://pastebin.com/raw.php?i=XXXXXXXX -O - | tr -d '\r' > /tmp/adblock/blacklist && echo "" >> /tmp/adblock/blacklist && echo success!
     
  20. IngoPan

    IngoPan Serious Server Member

    Whats the reason why i can´t see pixelserv in the ps task list? Also a killall pixelserv doesn´t work=no such task.
    Whats the take here?

    Thanks.

    @edit: Seems like i can´t assign a proper ip?
    pixelserv 192.168.2.252 -f
    pixelserv[2071]: pixelserv version: V35.HZ12WIP1 compiled: Jul 6 2015 19:20:09 options: 192.168.2.252 -f
    pixelserv[2071]: Abort: Cannot assign requested address - :192.168.2.252:80
     
  21. WaLLy3K

    WaLLy3K Networkin' Nut Member

    Is your primary subnet running on 192.168.2.X, opposed to 192.168.1.X? Also make sure that 252 isn't already assigned to another device.
     
  22. IngoPan

    IngoPan Serious Server Member

    Yes it is. I am on AsusWRT7Merlin here. But running pixelserv manually should work ,no? is there any working solution of this for AsusWRT?
     
  23. jerrm

    jerrm Network Guru Member

    AsusWRT's admin web server grabs port 80 on all interfaces. Either change the port for the admin web server, or run pixelserv on another port and use iptables to redirect the traffic.
     
    HunterZ likes this.
  24. IngoPan

    IngoPan Serious Server Member

    How do i change the webmin port please?
    In case i change the webmin port to something different, how do i run/start pixelserv please from commandline?

    Oh, and is there a live test i could do to see if pixelserv works "in the wild"?
     
    Last edited: Mar 18, 2016
  25. jerrm

    jerrm Network Guru Member

    Don't know, should be an option somehwhere in the GUI, but I don't use AsusWRT.

    Easiest way to test pixelserv is to query the statistics: http://pix.el..ser.vip/servstats
     
  26. IngoPan

    IngoPan Serious Server Member

    Ha, got it working! Its running on 192.168.2.1 port 80 now.
    Would you consider this as working ? Not so sure here:

    [​IMG]
     
    Last edited: Mar 19, 2016
  27. HunterZ

    HunterZ Network Guru Member

    iptables does the routing. The adblock script sets up the iptables rules for you. Not sure if it can do it on Merlin though.
     
  28. IngoPan

    IngoPan Serious Server Member

    Does the above pic i took look right or not? At least its processing some stuff ;)
     
  29. HunterZ

    HunterZ Network Guru Member

    Yes it looks like it's handling stuff. It doesn't look like SSL (port 443) is being redirected to it, though.
     
    IngoPan likes this.
  30. mstombs

    mstombs Network Guru Member

    In Asuswrt Merlin you can change the web gui to a non standard https port, say 4343. I then recommend Firefox over Chrome to access the web gui. No iptables then needed. I use this HZ binary on an N66 with Asuswrt merlin 380.57

     
  31. theoctavist

    theoctavist Reformed Router Member

    damn it
    Code:
     Blocklist generated - 796 seconds
    ADBLOCK[19451]: 202195 unique hosts to block
    ADBLOCK[19451]: Setting up 192.168.1.254 netmask 255.255.255.0 on br0:adblk
    ADBLOCK[19451]: Setting up pixelserv on 192.168.1.254
    ADBLOCK[19451]: /mnt/addons/adblock/adblock.sh: line 1149: /mnt/addons/adblock/pixelserv: Permission denied
    ADBLOCK[19451]: Writing File /etc/dnsmasq.custom
    ADBLOCK[19451]: CONF file /etc/dnsmasq.custom changed
    ADBLOCK[19451]: Restarting dnsmasq
     
  32. WaLLy3K

    WaLLy3K Networkin' Nut Member

    "chmod 755 /mnt/addons/adblock/pixelserv" ?
     
    theoctavist likes this.
  33. theoctavist

    theoctavist Reformed Router Member

    thanks man, partly there.. now I get this in the adblock gui... is the structure supposed to be adblock/pixelserv, or is the pixelserv binary supposed to be in the same folder as the adblock.sh script? i did a manual install so I think I may have copied the wrong files in there or something, i dunno.

    Code:
    ERROR: No response from pixelserv...
    pixelserv is not runnng on router for 192.168.1.254 
    also getting the error that blocking is down.
    here is my config info
    Code:
    ##
    
    # The only required line in the config file
    SOURCES=""   
    
    # a few example options
    #dnsmasq_logqueries=1
    #web_refreshtime=300
    #web_reportlines=200
    #LISTMODE=OPTIMIZE
    
    
    ## Remove comments from below lists to enable them
    
    ## Sources (uncomment desired blocklists) [must be compatible to the hosts file format!]
    ## MVPS HOSTS (~600k) [default]:
    SOURCES="$SOURCES http://winhelp2002.mvps.org/hosts.txt"
    
    ## pgl.yoyo.org (~70k) [default]:
    SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"
    
    ## AdAway mobile ads (~20k):
    SOURCES="$SOURCES http://adaway.org/hosts.txt"
    
    ## hpHosts ad/tracking servers (~400k):
    SOURCES="$SOURCES http://hosts-file.net/ad_servers.txt "
    
    ## The Cameleon Project (~600k):
    SOURCES="$SOURCES http://sysctl.org/cameleon/hosts"
    
    ## hpHosts ad/tracking/malicious servers (~6M! replaces hpHosts ad/tracking list):
    SOURCES="$SOURCES http://hosts-file.net/download/hosts.txt http://hosts-file.net/hphosts-partial.txt"
    
    ## MalwareDomainList.com (~40k):
    SOURCES="$SOURCES http://www.malwaredomainlist.com/hostslist/hosts.txt "
    
     
  34. WaLLy3K

    WaLLy3K Networkin' Nut Member

    Binary is supposed to be in the adblock folder, the same folder that contains adblock.sh.

    Your config appears to be okay.
     
    theoctavist likes this.
  35. theoctavist

    theoctavist Reformed Router Member

    working now, thank you buddy. the only problem now is that I cannot get any https hosts file in the config (to where they work anyway)
     
  36. WaLLy3K

    WaLLy3K Networkin' Nut Member

    It sounds like you've got it working now, which is good progress!

    You've mentioned elsewhere that you run a Linksys 2500v3, which is a MIPS router. From what I've read, the version of WGET bundled into MIPS based Tomato builds do not support HTTPS.

    If you did end up getting Entware working, you could try downloading WGET using ipkg/opkg and seeing if that will work.
     
    theoctavist likes this.
  37. koitsu

    koitsu Network Guru Member

    wget on Entware-ng and Entware does support HTTPS/SSL. Optware unknown.
     
  38. theoctavist

    theoctavist Reformed Router Member

    yep, entware is my next step!
     
  39. jerrm

    jerrm Network Guru Member

    Optware does as well, or even just installing Opt/Entware OpenSSL will enable https for busybox wget.
     
    WaLLy3K likes this.
  40. koitsu

    koitsu Network Guru Member

    I cannot confirm that in multiple ways. For example, I see nothing about the Busybox wget binary that makes use of relevant ELF linker flags that could potentially load something from /opt/lib instead of /lib (i.e. there is no RPATH):

    Code:
    root@gw:/tmp/home/root# /usr/bin/wget -O- 'https://www.google.com/'
    wget: not an http or ftp url: https://www.google.com/
    
    root@gw:/tmp/home/root# ldd /usr/bin/wget
            libcrypt.so.0 => /lib/libcrypt.so.0 (0x2aabf000)
            libm.so.0 => /lib/libm.so.0 (0x2aae3000)
            libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2ab01000)
            libc.so.0 => /lib/libc.so.0 (0x2ab20000)
            ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000)
    
    root@gw:/tmp/home/root# objdump -x /usr/bin/wget
    
    /usr/bin/wget:     file format elf32-tradlittlemips
    /usr/bin/wget
    architecture: mips:isa32, flags 0x00000112:
    EXEC_P, HAS_SYMS, D_PAGED
    start address 0x00407ca0
    
    Program Header:
        PHDR off    0x00000034 vaddr 0x00400034 paddr 0x00400034 align 2**2
             filesz 0x000000e0 memsz 0x000000e0 flags r-x
      INTERP off    0x00000114 vaddr 0x00400114 paddr 0x00400114 align 2**0
             filesz 0x00000014 memsz 0x00000014 flags r--
        LOAD off    0x00000000 vaddr 0x00400000 paddr 0x00400000 align 2**16
             filesz 0x000b0b3c memsz 0x000b0b3c flags r-x
        LOAD off    0x000b0b3c vaddr 0x004c0b3c paddr 0x004c0b3c align 2**16
             filesz 0x00003168 memsz 0x0000618c flags rw-
    DYNAMIC off    0x00000128 vaddr 0x00400128 paddr 0x00400128 align 2**2
             filesz 0x000000f0 memsz 0x000000f0 flags rwx
       STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**2
             filesz 0x00000000 memsz 0x00000000 flags rwx
        NULL off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**2
             filesz 0x00000000 memsz 0x00000000 flags ---
    
    Dynamic Section:
      NEEDED               libcrypt.so.0
      NEEDED               libm.so.0
      NEEDED               libgcc_s.so.1
      NEEDED               libc.so.0
      INIT                 0x00407c20
      FINI                 0x0049888c
      HASH                 0x00400218
      STRTAB               0x004048b0
      SYMTAB               0x004016c0
      STRSZ                0x00002d02
      SYMENT               0x00000010
      MIPS_RLD_MAP         0x004c2dac
      DEBUG                0x00000000
      PLTGOT               0x004c2db0
      MIPS_RLD_VERSION     0x00000001
      MIPS_FLAGS           0x00000002
      MIPS_BASE_ADDRESS    0x00400000
      MIPS_LOCAL_GOTNO     0x000000a9
      MIPS_SYMTABNO        0x0000031f
      MIPS_UNREFEXTNO      0x0000001a
      MIPS_GOTSYM          0x0000000c
      VERNEED              0x00407bf0
      VERNEEDNUM           0x00000001
      VERSYM               0x004075b2
    
    Version References:
      required from libgcc_s.so.1:
        0x0b792650 0x00 03 GCC_3.0
        0x0d696910 0x00 02 GLIBC_2.0
    private flags = 50001007: [abi=O32] [mips32] [not 32bitmode] [noreorder] [PIC] [CPIC]
    
    Sections:
    Idx Name          Size      VMA       LMA       File off  Algn
      0 .interp       00000014  00400114  00400114  00000114  2**0
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      1 .dynamic      000000f0  00400128  00400128  00000128  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      2 .hash         000014a8  00400218  00400218  00000218  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      3 .dynsym       000031f0  004016c0  004016c0  000016c0  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      4 .dynstr       00002d02  004048b0  004048b0  000048b0  2**0
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      5 .gnu.version  0000063e  004075b2  004075b2  000075b2  2**1
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      6 .gnu.version_r 00000030  00407bf0  00407bf0  00007bf0  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      7 .init         00000078  00407c20  00407c20  00007c20  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
      8 .text         0008fb0c  00407ca0  00407ca0  00007ca0  2**4
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
      9 .MIPS.stubs   000010e0  004977ac  004977ac  000977ac  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
    10 .fini         00000050  0049888c  0049888c  0009888c  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
    11 .rodata       00018258  004988e0  004988e0  000988e0  2**4
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
    12 .eh_frame     00000004  004b0b38  004b0b38  000b0b38  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
    13 .ctors        00000008  004c0b3c  004c0b3c  000b0b3c  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    14 .dtors        00000008  004c0b44  004c0b44  000b0b44  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    15 .jcr          00000004  004c0b4c  004c0b4c  000b0b4c  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    16 .data.rel.ro  00002130  004c0b50  004c0b50  000b0b50  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    17 .data         0000012a  004c2c80  004c2c80  000b2c80  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    18 .rld_map      00000004  004c2dac  004c2dac  000b2dac  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    19 .got          00000ef0  004c2db0  004c2db0  000b2db0  2**4
                      CONTENTS, ALLOC, LOAD, DATA
    20 .sdata        00000004  004c3ca0  004c3ca0  000b3ca0  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    21 .sbss         00000037  004c3ca4  004c3ca4  000b3ca4  2**2
                      ALLOC
    22 .bss          00002fe8  004c3ce0  004c3ce0  000b3ca4  2**4
                      ALLOC
    23 .mdebug.abi32 00000000  00000022  00000022  000b3ca4  2**0
                      CONTENTS, READONLY
    SYMBOL TABLE:
    no symbols
    
    
    Comparatively, Entware-ng (ignore the IPv6 failure, that's intentional given my setup):

    Code:
    root@gw:/tmp/home/root# /opt/bin/wget --no-check-certificate -O- 'https://www.google.com/'
    --2016-03-27 20:51:56--  https://www.google.com/
    Resolving www.google.com... 2607:f8b0:400e:c01::68, 173.194.79.104, 173.194.79.147, ...
    Connecting to www.google.com|2607:f8b0:400e:c01::68|:443... failed: Network is unreachable.
    Connecting to www.google.com|173.194.79.104|:443... connected.
    WARNING: cannot verify www.google.com's certificate, issued by 'CN=Google Internet Authority G2,O=Google Inc,C=US':
      Unable to locally verify the issuer's authority.
    HTTP request sent, awaiting response... 200 OK
    Length: unspecified [text/html]
    Saving to: 'STDOUT'
    {snip content}
    
    root@gw:/tmp/home/root# ldd /opt/bin/wget
            libpcre.so.1 => /opt/lib/libpcre.so.1 (0x2aac0000)
            libssl.so.1.0.0 => /opt/lib/libssl.so.1.0.0 (0x2ab12000)
            libcrypto.so.1.0.0 => /opt/lib/libcrypto.so.1.0.0 (0x2ab7d000)
            libdl.so.1 => /opt/lib/libdl.so.1 (0x2acf9000)
            libz.so.1 => /opt/lib/libz.so.1 (0x2ad0d000)
            libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x2ad31000)
            libc.so.1 => /opt/lib/libc.so.1 (0x2ad57000)
            ld-uClibc.so.1 => /opt/lib/ld-uClibc.so.0 (0x2aaa8000)
    
    root@gw:/tmp/home/root# objdump -x /opt/bin/wget
    
    /opt/bin/wget:     file format elf32-tradlittlemips
    /opt/bin/wget
    architecture: mips:isa32r2, flags 0x00000112:
    EXEC_P, HAS_SYMS, D_PAGED
    start address 0x004064b0
    
    Program Header:
        PHDR off    0x00000034 vaddr 0x00400034 paddr 0x00400034 align 2**2
             filesz 0x00000100 memsz 0x00000100 flags r-x
      INTERP off    0x00000134 vaddr 0x00400134 paddr 0x00400134 align 2**0
             filesz 0x00000018 memsz 0x00000018 flags r--
    0x70000000 off    0x0000014c vaddr 0x0040014c paddr 0x0040014c align 2**2
             filesz 0x00000018 memsz 0x00000018 flags r--
        LOAD off    0x00000000 vaddr 0x00400000 paddr 0x00400000 align 2**16
             filesz 0x00073d20 memsz 0x00073d20 flags r-x
        LOAD off    0x00074000 vaddr 0x00484000 paddr 0x00484000 align 2**16
             filesz 0x0000348c memsz 0x00008838 flags rw-
    DYNAMIC off    0x00000164 vaddr 0x00400164 paddr 0x00400164 align 2**2
             filesz 0x00000148 memsz 0x00000148 flags rwx
       STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4
             filesz 0x00000000 memsz 0x00000000 flags rw-
        NULL off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**2
             filesz 0x00000000 memsz 0x00000000 flags ---
    
    Dynamic Section:
      NEEDED               libpcre.so.1
      NEEDED               libssl.so.1.0.0
      NEEDED               libcrypto.so.1.0.0
      NEEDED               libdl.so.1
      NEEDED               libz.so.1
      NEEDED               libgcc_s.so.1
      NEEDED               libc.so.1
      RPATH                /opt/lib
      INIT                 0x004033ac
      FINI                 0x00451730
      HASH                 0x004002ac
      STRTAB               0x00401cb0
      SYMTAB               0x00400b30
      STRSZ                0x00000c32
      SYMENT               0x00000010
      MIPS_RLD_MAP         0x00487470
      DEBUG                0x00000000
      PLTGOT               0x00487480
      REL                  0x00402b44
      RELSZ                0x00000038
      RELENT               0x00000008
      MIPS_RLD_VERSION     0x00000001
      MIPS_FLAGS           0x00000002
      MIPS_BASE_ADDRESS    0x00400000
      MIPS_LOCAL_GOTNO     0x00000002
      MIPS_SYMTABNO        0x00000118
      MIPS_UNREFEXTNO      0x00000022
      MIPS_GOTSYM          0x00000118
      PLTREL               0x00000011
      JMPREL               0x00402b7c
      PLTRELSZ             0x00000830
      DT_MIPS_PLTGOT       0x00484010
      VERNEED              0x00402b14
      VERNEEDNUM           0x00000001
      VERSYM               0x004028e2
    
    Version References:
      required from libgcc_s.so.1:
        0x0d696910 0x00 03 GLIBC_2.0
        0x0b792650 0x00 02 GCC_3.0
    private flags = 70001005: [abi=O32] [mips32r2] [not 32bitmode] [noreorder] [CPIC]
    
    Sections:
    Idx Name          Size      VMA       LMA       File off  Algn
      0 .interp       00000018  00400134  00400134  00000134  2**0
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      1 .reginfo      00000018  0040014c  0040014c  0000014c  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA, LINK_ONCE_SAME_SIZE
      2 .dynamic      00000148  00400164  00400164  00000164  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      3 .hash         00000884  004002ac  004002ac  000002ac  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      4 .dynsym       00001180  00400b30  00400b30  00000b30  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      5 .dynstr       00000c32  00401cb0  00401cb0  00001cb0  2**0
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      6 .gnu.version  00000230  004028e2  004028e2  000028e2  2**1
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      7 .gnu.version_r 00000030  00402b14  00402b14  00002b14  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      8 .rel.dyn      00000038  00402b44  00402b44  00002b44  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
      9 .rel.plt      00000830  00402b7c  00402b7c  00002b7c  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
    10 .init         00000048  004033ac  004033ac  000033ac  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
    11 .text         0004e330  00403400  00403400  00003400  2**4
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
    12 .fini         00000038  00451730  00451730  00051730  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
    13 .rodata       00021520  00451770  00451770  00051770  2**4
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
    14 .eh_frame     00000004  00472c90  00472c90  00072c90  2**2
                      CONTENTS, ALLOC, LOAD, READONLY, DATA
    15 .plt          00001080  00472ca0  00472ca0  00072ca0  2**5
                      CONTENTS, ALLOC, LOAD, READONLY, CODE
    16 .ctors        00000008  00484000  00484000  00074000  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    17 .dtors        00000008  00484008  00484008  00074008  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    18 .got.plt      00000420  00484010  00484010  00074010  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    19 .data         00003040  00484430  00484430  00074430  2**4
                      CONTENTS, ALLOC, LOAD, DATA
    20 .rld_map      00000004  00487470  00487470  00077470  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    21 .got          00000008  00487480  00487480  00077480  2**4
                      CONTENTS, ALLOC, LOAD, DATA
    22 .sdata        00000004  00487488  00487488  00077488  2**2
                      CONTENTS, ALLOC, LOAD, DATA
    23 .sbss         00000048  00487490  00487490  0007748c  2**3
                      ALLOC
    24 .bss          00005358  004874e0  004874e0  0007748c  2**4
                      ALLOC
    25 .comment      00000038  00000000  00000000  0007748c  2**0
                      CONTENTS, READONLY
    26 .pdr          00005820  00000000  00000000  000774c4  2**2
                      CONTENTS, READONLY
    27 .gnu.attributes 00000010  00000000  00000000  0007cce4  2**0
                      CONTENTS, READONLY
    28 .mdebug.abi32 00000000  00000000  00000000  0007ccf4  2**0
                      CONTENTS, READONLY
    SYMBOL TABLE:
    no symbols
    
    root@gw:/tmp/home/root# opkg list-installed | grep -i ssl
    libopenssl - 1.0.2f-1
    
    root@gw:/tmp/home/root# opkg files libopenssl
    Package libopenssl (1.0.2f-1) is installed on root and has the following files:
    /opt/lib/libssl.so.1.0.0
    /opt/lib/libcrypto.so.1.0.0
    
     
  41. theoctavist

    theoctavist Reformed Router Member

    got adblock running fine now, and just want to make one more addition. use a HTTPS hosts file. I have entware-ng going, so what more should I do?
     
  42. jerrm

    jerrm Network Guru Member

    It doesn't link the libraries, it uses the opensssl executable's s_client app to establish the connection. As long as the openssl found in the path supports s_client, bb wget will support https. This is the reason it works under ARM - Shibby added s_client back to ARM's openssl, but did not under MIPS. The busbox versions/options are the same.

    It needs to be a busybox version from the past year or so.
     
    Last edited: Mar 28, 2016
    koitsu likes this.
  43. WaLLy3K

    WaLLy3K Networkin' Nut Member

    From that discussion above, I'd say that running opkg install openssl from SSH or Tools > System Commands should do the trick! (If opkg isn't a valid command, use ipkg. It's my understanding that Entware uses one while Optware (What I'm used to) uses the other).
     
  44. theoctavist

    theoctavist Reformed Router Member

    ok seems to be working now,.. but where do i issue the --no--check certificate command?

    in front of the hosts file url?

    Code:
    ERROR: cannot verify raw.githubusercontent.com's certificate, issued by 'CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US':
      Unable to locally verify the issuer's authority.
    To connect to raw.githubusercontent.com insecurely, use `--no-check-certificate'.
     
  45. jerrm

    jerrm Network Guru Member

    Looks like you installed wget from opt/ent-ware.

    Install the ca-certificates package.
     
  46. theoctavist

    theoctavist Reformed Router Member

    you are a fount of knowledge dear man. thank you
     
  47. theoctavist

    theoctavist Reformed Router Member

    one last question(and I think ive got it sorted). in the adblock. ini , the dnsmasq log queries-=1 . that means that logging is enabled, yes? if so, i wonder why the webgui indicates otherwise
     
  48. WaLLy3K

    WaLLy3K Networkin' Nut Member

    Make sure adblock.ini reads:

    Code:
    dnsmasq_logqueries=1
    It should then be reflected on the adblock webpage (/user/adblock.sh) by saying "logging: up"
     
  49. crusher9

    crusher9 Serious Server Member

    easiest way to block youtube ads for all devices is to enable "Intercept DNS port (UDP 53)" in DNS-settings
    ... and use the host-file of course
     
  50. IngoPan

    IngoPan Serious Server Member

    From within the Tomato GUI or where to intercept DNS please? Use the host file means enable adblocking from this script? Will it block Facebook/twitter ads, too? It´s so annoying seeing all those Amazon Ads within fb.
     
  51. WaLLy3K

    WaLLy3K Networkin' Nut Member

    Advanced > DHCP/DNS > Intercept DNS port (UDP 53)
     
    IngoPan likes this.
  52. IngoPan

    IngoPan Serious Server Member

    Is this version working with https ,too? Took a snapshot of my running one. Is this setup valid for blocking https ads?
    [​IMG]
    [​IMG]
     
    Last edited: Apr 4, 2016
  53. HunterZ

    HunterZ Network Guru Member

    Looks like you've blocked 27 HTTPS requests, so I'd say yes. I don't know anything about that version of pixelserv though.
     
  54. meazz1

    meazz1 LI Guru Member

    I installed it on my Rt-N16 router running shibby v132 firmware. It's running but there's no "adblock" menu in my router. In my other ARM router, I see the GUI menu added.
    Can a menu be added in the RT-N16 also?
     
  55. HunterZ

    HunterZ Network Guru Member

    meazz1 likes this.
  56. meazz1

    meazz1 LI Guru Member

  57. jsnepo

    jsnepo Serious Server Member

    So I got this working perfectly. How can I make this work on the 2nd bridge (br1) that I set up?
     
  58. theoctavist

    theoctavist Reformed Router Member

    @jerrm is this my problem too?
    ADBLOCK[1828]: Running as /opt/bin/adblock.sh
    ADBLOCK[1828]: Using config file /opt/etc/adblock.ini
    ADBLOCK[1828]: Ignoring extra config file /opt/adblock.ini
    ADBLOCK[1828]: Requested list mode is HOST
    ADBLOCK[1828]: Enabling dnsmasq logging
    ADBLOCK[1828]: Logging to syslog
    ADBLOCK[1828]: Creating web link /www/user/adblock.sh
    ADBLOCK[1828]: Web interface should be available at http://192.168.1.1/user/adblock.sh
    ADBLOCK[1828]: Adding tomato menu item
    ADBLOCK[1828]: Download starting
    ADBLOCK[1828]: Downloading: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn-social/hosts
    Connecting to raw.githubusercontent.com (23.235.39.133:443)
    wget: error getting response: Connection reset by peer
    ADBLOCK[1828]: Failed: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn-social/hosts
    ADBLOCK[1828]: No source files found
     
  59. koitsu

    koitsu Network Guru Member

    This looks more like a SSL (protocol) version / TLS version compatibility issue (i.e. the wget binary is using an OpenSSL library that is either old or lacks TLS v1.2 or similar extension capabilities). Lots of places are doing this now (including CloudFlare). Let's see what the compatibility is using Qualys' SSL Test, which tests using several different SSL libraries, ciphers, and extensions, shall we?

    https://www.ssllabs.com/ssltest/analyze.html?d=raw.githubusercontent.com

    Looks like the server will drop the connection it if the SNI header/extension isn't provided, ditto with forward secrecy. The server is obviously very "cipher-centric" as well. Otherwise it seems to support even old unpatched OpenSSL.

    Debugging this (to determine root cause) is sometimes painful and almost always requires extensive knowledge of SSL (not just "connect to TCP port 443" -- I'm talking about the actual SSL/TLS protocol that flows underneathe).

    This is the problem with the "just grab a wget binary from somewhere that was built with SSL support" approach: OpenSSL is horribly insecure and more and more places running HTTPS/SSL services are beginning to reject less secure protocol versions, and such binaries are guaranteed to be outdated eventually. With things like Heartbleed, DROWN, and tons of others, you're really better off using something like Entware-ng and simply referring to the Entware-ng wget binary (/opt/bin/wget) + updating your Entware-ng packages (opkg update && opkg upgrade) regularly.

    For example, we can see that SSLv2 and SSLv3 are explicitly rejected (this would be "TCP connection reset"), but TLSv1 works fine:

    Code:
    $ echo | openssl s_client -connect raw.githubusercontent.com:443 -showcerts -ssl2 -msg
    CONNECTED(00000004)
    >>> SSL 2.0 [length 0022], CLIENT-HELLO
        01 00 02 00 09 00 00 00 10 07 00 c0 03 00 80 01
        00 80 38 40 02 10 36 33 15 1f fe 61 44 af 9c eb
        a6 76
    7181:error:1407F0E5:SSL routines:SSL2_WRITE:ssl handshake failure:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s2_pkt.c:403:
    
    $ echo | openssl s_client -connect raw.githubusercontent.com:443 -showcerts -ssl3 -msg
    CONNECTED(00000004)
    >>> SSL 3.0 Handshake [length 0055], ClientHello
        01 00 00 51 03 00 57 25 31 76 72 ad c5 d2 ae 76
        35 90 66 a2 ad 8b 19 56 0a 81 1f ee 3d ee 2b 75
        c1 cf 02 5a 9c 2d 00 00 2a 00 88 00 87 00 84 00
        39 00 38 00 35 00 16 00 13 00 0a 00 45 00 44 00
        41 00 33 00 32 00 2f 00 05 00 04 00 15 00 12 00
        09 00 ff 01 00
    <<< SSL 3.0 Alert [length 0002], fatal handshake_failure
        02 28
    7186:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_pkt.c:1146:SSL alert number 40
    7186:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_pkt.c:572:
    
    It could also be something as simple as github having a broken server at some point. :) But if it's like this for you consistently, for days, that's pretty unlikely.
     
  60. leandroong

    leandroong LI Guru Member

  61. theoctavist

    theoctavist Reformed Router Member

    thank you @koitsu (again). I think if I can get tomato ware going, i will be ok. when i tried to extract the files to my usb drive (winscp) , there were a lot of errors I guess due to the existence of entware on the same drive. I thought I saw lance say that the two could exist in unison, but should I put tomatoware on a different partition? (entware-ng exists on tmp/mnt/"addons"
     
  62. leandroong

    leandroong LI Guru Member

    1. winscp
    1. remove old tomatoware *.tgz
    2. download latest tomare *.tgz
    3. delete all files and folders, except the new tomatoware*.tgz
    4. extract and ur done
     
  63. theoctavist

    theoctavist Reformed Router Member

    thanks. appreciated. so entware-ng must go, yeah?
     
  64. koitsu

    koitsu Network Guru Member

    I don't know anything about "Tomatoware", but I can assure you Entware-ng's wget package works just fine with HTTPS / that URL. If "Tomatoware"'s wget works too, awesome.
     
  65. theoctavist

    theoctavist Reformed Router Member

    thank you @koitsu Perhaps I am misdiagnosing the error I am getting.
     
  66. theoctavist

    theoctavist Reformed Router Member

    Code:
    Download starting
    ADBLOCK[2630]: Downloading: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn-social/hosts
    Connecting to raw.githubusercontent.com (23.235.39.133:443)
    wget: error getting response: Connection reset by peer
    ADBLOCK[2630]: Failed: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn-social/hosts
    ADBLOCK[2630]: No source files found
    
     
  67. leandroong

    leandroong LI Guru Member

    there is no problem for the source site for sure, I can view the file, in my location
    note: Try it yourself, see if you can view that on firefox browser
     
  68. tmr250z

    tmr250z Network Guru Member

    @koitsu I was getting the failed download error (minus the "No source files found" part) for that link as well with tomato's default wget, so I installed Entware-ng's wget package, tried the link again, running "force" from the adblock web interface. I still get a failed download error, only now the error says it cannot verify raw.githubusercontent.com's certificate. What am I doing wrong?

    Code:
    ADBLOCK[11314]: Downloading: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn-social/hosts
    --2016-05-03 15:59:52--  https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn-social/hosts
    Resolving raw.githubusercontent.com... 23.235.40.133
    Connecting to raw.githubusercontent.com|23.235.40.133|:443... connected.
    ERROR: cannot verify raw.githubusercontent.com's certificate, issued by 'CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US':
      Unable to locally verify the issuer's authority.
    To connect to raw.githubusercontent.com insecurely, use `--no-check-certificate'.
    ADBLOCK[11314]: Failed: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/gambling-porn-social/hosts
     
    Last edited: May 3, 2016
  69. jerrm

    jerrm Network Guru Member

    Install the ca-certificates package.
     
    koitsu likes this.
  70. koitsu

    koitsu Network Guru Member

    You're doing nothing wrong. In several programs by default, when using SSL, the client (wget/OpenSSL) must verify the validity (specifically the CA signing) of the server's SSL certificate. To do this, the client needs to have a whole bunch of CA signatures/etc. available locally. Without these, the client cannot do verification of the CA.

    The solution is to do what @jerrm said (opkg install ca-certificates), or modify the script to use wget --no-check-certificates (the argument causes wget to disable CA verification).
     
  71. theoctavist

    theoctavist Reformed Router Member

    fantastic! worked a treat. thanks koitsu! I wonder what was different between say 3 days ago (when adblock started acting up) and a week prior, when it was working fine?
     
  72. tmr250z

    tmr250z Network Guru Member

    Yes, that did it! Thanks @jerrm and @koitsu
     
    GLuDeRo likes this.
  73. theoctavist

    theoctavist Reformed Router Member

    one thing i noticed @HunterZ . when I amended my .ini to match yours, photos are not displaying properly on facebook. i cannot for the life of me figure out where the pictures originate(i have facebook.com whitelisted). any ideas?
     
  74. Filpos

    Filpos Reformed Router Member

    Should this script be able to block Spotify ads when using it thru Chromecast?
     
  75. HunterZ

    HunterZ Network Guru Member

    I think you meant to post in the other thread?

    I've had to manually whitelist some sites. You can usually figure out where the image comes from by inspecting the site and/or looking at the adblock status page to see what is being blocked.
     
  76. Filpos

    Filpos Reformed Router Member

    I used the scripts and configs from the 1st post and put the adblock in the jffs.
    Seems to work but after awhile some pages load really really slow if at all and I have to disable adblock to get those pages to load or to load faster.
    How would i go about troubleshooting what is causing this?

    My set:
    Tomato Firmware 1.28.0000 MIPSR2-136 K26 Max
    Model Asus RT-N12 B1 (actually it is D1)
    Chipset Broadcom BCM53572 chip rev 1 pkg 8
    CPU Freq 300MHz
    Flash Size 8MB
     
  77. pharma

    pharma Network Guru Member

  78. Filpos

    Filpos Reformed Router Member

    Cheers. I missed that totally. Gonna check that out.
     
  79. jan.n

    jan.n LI Guru Member

    Resolved, thank you everyone, tomato is just awsome!
     
    Last edited: Jul 3, 2016
  80. Frequenzy

    Frequenzy Networkin' Nut Member

    how do you run adblock with pixelserv on merlin, im trying to use jermms script but it doesnt work. thanks
     
  81. mstombs

    mstombs Network Guru Member

    I basically use a variant of the original scripts - I am currently using Entware pixelserv-tls with a custom dnsmasq config. Some tricks needed to use manually generated blocklist on usb, which will not be available first time dnsmasq runs. Not at home at the moment, can't post details for a couple of weeks.
     
  82. Frequenzy

    Frequenzy Networkin' Nut Member

    once you have time can would you mind sharing how its done on merlin. thanks.
     
  83. RypeDub

    RypeDub New Member Member

    Hey everyone!

    Thank you for this fantastic tutorial on how to get a whole house adblocked.

    However, I ran into this issue:
    I have no idea what this means and I got it working just a second ago. Only when I messed with the sources do I now have this issue and i've even restarted the whole process.

    Do I need to reboot my router for some reason?

    Also, my ONLY reason for wanting to set this up was so that on KissAnime.to I could watch Naruto Shippuden on my Samsung UDH 4k Smart TV's web browser. Currently, EVERYTIME you activate ANY link, or even click on ANY space on the webpages: a popup comes up and this hinders the ability to go full screen and even causes crashing.

    When I got it working for a second, it blocked some of the visual ads, but it didn't block the popup window. Could I have someone instruct me on how I can add blocking for that one particular website so I can get on with my life lol

    Thank you all again for this amazing tutorial!!


    I have an ASUS N900 RT-N66R (which is compatible with anything on the internet labeled as RT-N66U <-- U = Universal | R = Regional (USA/UK/Germany/etc.)

    The regional versions have limitations imposed in the firmware's depending on the area the router is being sold in order to be compliant with the FCC and other regulations. The Universal Firmware can be flashed onto ANY Regional Router and turn it into a Universal version.

    IS IT YOUR responsibility to figure out what transmission rates and other limitations and regulations are implemented in your area. No one here is responsible for your consequences. Now go have fun hacking!
     
  84. dbareis

    dbareis Networkin' Nut Member

    Hi,

    Just started using this tool and so far it seems to be great in general and probably better than anything else around.

    This thread hasn't been updated in a while but I'll list some bugs/features anyway in case they can be fixed. Otherwise I'll eventually try to work out whats going on myself :)
    1. You can change the "blacklist" file and then run "/cifs1/adblock/adblock.sh" which will terminate without updating the blocklist as it only looks at the network list caches. It probably has the same bug for the "whilelist" file.
    2. At least in my opinion it should ignore blank lines and have allow for commenting in the black & white file lists.
    3. It would be nice if the file lists ignored Windows CRLF terminations or at least reported them as causing problems when the script is run.
    4. I noticed some other issues but I'll see how the current list goes first :)
     
    Last edited: Nov 25, 2017
  85. rs232

    rs232 Network Guru Member

    Please ignore this post, wrong thread :(
     
  86. plikmuny

    plikmuny New Member Member

    Can you share me your Sources ? i have my Pixelserv blocking only 52105 Hosts. No matter how many host sources i add, it doesnt block further.
    By the way i am using "Adblock - not so lean" method and i have OPTIMIZE mode enabled. Should i Change it to HOST mode to get better result ? i have installed it to jffs internal on R7000 running Tomato Shibby latest Version. Waiting for your reply.
    Thanks.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice