1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Script: Clean, Lean and Mean Adblocking

Discussion in 'Tomato Firmware' started by haarp, Apr 23, 2013.

  1. haarp

    haarp LI Guru Member

    Hello,
    I've been looking into blocking ads on Tomato firmwares. There are already scripts for this, such as xcooling's script and ALL-U-NEED Ad Blocking, but they do not suit my needs. They're bloated, unreadable, slow, wasteful and don't offer the functionality I want.

    I therefore wrote my own script, while taking ideas from these two and this DD-WRT script. I then optimized the heck out of it.

    Features:
    • Takes public blocklists for known ad hosts and redirects them via DNS poisoning
    • pixelserv (optional) through a second router IP (Web GUI on port 80 still works!)
    -> e.g. router is 192.168.0.1, but also responds as 192.168.0.254 with pixelserv
    -> this serves transparent pixels instead of causing error messages on blocked ads
    • Or if pixelserv is not desired, redirect to 0.0.0.0, which will also kill ads (but might produce error messages)
    • Does not interfere with normal dnsmasq operation
    -> does not try to "optimize" it (that's what the "Custom configuration" box on the web GUI is for, people!)
    -> does not break Tomato's ability to restart dnsmasq should it crash
    • Additional blocklist sources can easily be added
    • Easy blacklist and whitelist
    • Very optimized: Updates as quickly and with as little CPU/memory usage as possible
    • Small and lean: Only does what it needs to do, then gets out of the way
    • Readable code
    Instructions:
    If you're using Adblock pre-v4.0, please remove it completely from your router and reboot it first!
    Note: If the paste procedure fails for you, try transferring the corresponding file manually using scp, WinSCP or something (the DD-WRT link up there has some WinSCP usage examples)
    • Verify that your Tomato supports custom dnsmasq configs (i.e. shows this line under Advanced->DHCP/DNS: "Note: The file /etc/dnsmasq.custom is also added to the end of Dnsmasq's configuration file if it exists.")
    • Set up some kind of non-volatile storage. This is up to you, options are JFFS, CIFS, SD card, USB and possibly more. Note the path.
    -> The simplest would probably be JFFS. Check this link
    -> I recommend other storage methods however, as JFFS is very limited in size (depends on which filter lists you'll ultimately use of course)
    • Designate a directory on your storage for adblock, e.g. /jffs/adblock/ (as seen by the router). Avoid spaces! This is the PREFIX.
    • Install pixelserv if desired (thread). Take the entire chunk of script in this link, adjust PREFIX at the top, paste it into the box on Tools->System and press Execute. This is also how you can update pixelserv in the future.
    If this fails, extract the pixelserv binary from the archive in the threadand manually transfer it to PREFIX/pixelserv
    • Install adblock.sh v4.5. Take the entire chunk of script in this link, adjust PREFIX at the top, paste it into the box on Tools->System and press Execute. This is also how you can update Adblock in the future.
    If this fails, paste the script from this link into a file and manually transfer it to PREFIX/adblock.sh
    • Install the config file. It will become PREFIX/config. Take the script below, adjust the config to your tastes, paste it into the box on Tools->System and press Execute. That's also how you can change the configuration in the future.
      Code:
      PREFIX="/cifs1/adblock/" ## adjust this!
      
      echo '
      ### Settings ###
      PIXEL_IP="254"    ## 0: disable pixelserv
              ## 1-254: last octet of IP to run pixelserv on (default=254)
      PIXEL_OPTS=""    ## additional options for pixelserv
      BRIDGE="br0"    ## bridge interface for pixelserv (default=br0)
      RAMLIST="0"    ## 1: keep blocklist in RAM (e.g. for small JFFS) (default=0)
      CONF="/etc/dnsmasq.custom"    ## dnsmasq custom config (must be sourced by dnsmasq!)
                      ## confused? then leave this be!
      
      
      ### Sources (uncomment desired blocklists) [must be compatible to the hosts file format!] ###
      ## MVPS HOSTS (~600k) [default]:
      SOURCES="$SOURCES http://winhelp2002.mvps.org/hosts.txt"
      ## pgl.yoyo.org (~70k) [default]:
      SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"
      ## Hosts File Project (~3M!):
      #SOURCES="$SOURCES http://hostsfile.mine.nu/Hosts"
      ## The Cameleon Project (~600k):
      #SOURCES="$SOURCES http://sysctl.org/cameleon/hosts"
      ## AdAway mobile ads (~20k):
      #SOURCES="$SOURCES http://adaway.sufficientlysecure.org/hosts.txt"
      ## hpHosts ad/tracking servers (~400k):
      #SOURCES="$SOURCES http://hosts-file.net/ad_servers.asp"
      ## hpHosts ad/tracking/malicious servers (~6M! replaces hpHosts ad/tracking list):
      #SOURCES="$SOURCES http://hosts-file.net/download/hosts.txt, http://hosts-file.net/hphosts-partial.asp"
      ## MalwareDomainList.com (~40k):
      SOURCES="$SOURCES http://www.malwaredomainlist.com/hostslist/hosts.txt"
      
      
      ### Blacklist additional sites ###
      ## (add hostnames inside the quotes, space-separated, without http://) ##
      BLACKLIST=""
      
      ### Whitelist sites from blocking ###
      ## (add hostnames inside the quotes, space-separated, without http://) ##
      WHITELIST="de.ign.com followerscounter.com redirectingat.com"
      
      ### Blacklist and Whitelist files (optional) ###
      ## create the files "blacklist" and "whitelist" with your hosts, one per line ##
      ## useful if you have many hosts in these (they generate faster aswell!) ##
      ' > "$PREFIX/config" && echo success
    • Ready for the first run! Paste this into the box on Tools->System, edit the PREFIX and press Execute: /YOUR/PREFIX/HERE/adblock.sh
    • The script also accepts a few command-line options:
    -> /YOUR/PREFIX/HERE/adblock.sh - default, update and enable adblocker
    -> /YOUR/PREFIX/HERE/adblock.sh force - force updating of filters, even if not outdated
    -> /YOUR/PREFIX/HERE/adblock.sh stop - disable the adblocker
    -> /YOUR/PREFIX/HERE/adblock.sh toggle - disable the adblocker if active, enable if inactive (perfect for the SES button on your WRT54G!)
    -> /YOUR/PREFIX/HERE/adblock.sh restart - restart adblocker (e.g. for config changes, script updates)
    • If you want to enable Adblock automatically when the router boots, just put /YOUR/PREFIX/HERE/adblock.sh into the WAN Up section on Administration->Scripts
    • If you want to have Adblock automatically update its filters, just put /YOUR/PREFIX/HERE/adblock.sh into one of the Custom commands on Administration->Scheduler (once every week should be enough)
    • Have fun! :)
    Notes:
    • This script was tested with TomatoUSB v1.28.8754 ND vpn3.6 on a WRT54GL.
    • Remember that the script, pixelserv and all data reside on PREFIX. If you have other means of accessing that storage, those will probably be more convenient than pasting into boxes on the web GUI.
    • The script will automatically block anything bound for the pixelserv IP that is not intended for pixelserv itself.
    • Subsequent updates will only fetch blocklists that have changed, this however only works when the source server runs http on port 80.
    • You will need 2x-3x as much non-volatile storage as all filters combined. If your storage is too small, set RAMLIST to 1. Obviously, you now need enough free RAM to hold the filters instead! Additionally, the script will now have to redownload the filters when the router reboots.
    • If you change settings or update the script, please run the script with the restart option afterwards!
    • I've you're having problems, check the router logs!
    Changelog:
    3.3 - Fixed small issue with IPv6 present
    3.4 - Added bridge interface selector and minor changes
    3.5 - Prevent multiple instances, block everything but pixelserv on pixel IP, add Malware source and minor changes
    3.6 - Now checks if blocklists are outdated
    3.6.1 - Added "force" option to ignore Last-Modified headers
    3.7 - Added "bigmem" mode
    4.0 - Fundamental changes, see this thread
    4.1 - Minor changes, more checks
    4.5 - Fixed lastmod for some hosts, added timeout to nc to avoid some bugs, updated pixelserv to v31, added blacklist and whitelist files, slightly less memory usage
     
  2. darkknight93

    darkknight93 Networkin' Nut Member

    :O you are reading my mind? Damn nice! I'll give it a shot :)

    Many many thanks!!

    [​IMG]


    Somehow something does no do what it should :/
    Code:
    ADBLOCK: Download starting
    ADBLOCK: Waiting for internet to come up...
    Connecting to winhelp2002.mvps.org (216.155.126.40:80)
    -                      1% |                               |  6723   0:01:25 ETAConnecting to pgl.yoyo.org (95.172.9.82:80)
    -                    100% |*******************************| 68203   0:00:00 ETA
    -                    100% |*******************************|   562k  0:00:00 ETA
    ADBLOCK: Download finished
    ADBLOCK: Setting up pixelserv on 10.10.1.250
    470
    ifconfig: bad address '470'
    Usage:/opt/adblock/pixelsrv/pixelserv [IP No/hostname (all)] [-u user ("nobody")] [-f response.bin] [-g name.gif]
    IP pixelserv
    10.10.1.250 470
    ADBLOCK: Generating /opt/adblock/dnsmasq.custom
    sed: unmatched ':'
    ADBLOCK: Config generated, 15858 unique hosts to block
    ADBLOCK: Restarting dnsmasq
    giving dnsmasq some time to start...
    
    dnsmasq.custom - seems to ignore the sed Parameter?
    Code:
     
     
    0.datacollector.coin.scribol.com
     
    0.r.msn.com
     
    005.free-counter.co.uk
     
    006.free-counter.co.uk
     
    007.free-counter.co.uk
     
    008.free-counter.co.uk
     
    008.free-counters.co.uk
     
    00fun.com
     
    011707160008.c.mystat-in.net
     
    061606084448.c.mystat-in.net
     
    070806142521.c.mystat-in.net
    ....
    
     
  3. darkknight93

    darkknight93 Networkin' Nut Member

    EDIT: I found it... Funky issue!

    My
    Code:
    redirip=$(ifconfig br0 | awk '/inet/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PIXELIP/")"
    
    gives a Output of TWO lines! 10.10.1 \n 240.

    I added the following beneath it:
    Code:
    .....redirip=$(ifconfig br0 | awk '/inet/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PIXELIP/")"
    redirip=$(echo $redirip | awk '{print $1}') to only geht the "10.10.1"
    
    Now it works :)

    EDIT: SWEEEET JESUS it works!
     
  4. haarp

    haarp LI Guru Member

    This is weird. Let me verify your change. Please give me the output of
    Code:
    ifconfig br0
     
  5. darkknight93

    darkknight93 Networkin' Nut Member

    br0 Link encap:Ethernet HWaddr ...
    inet addr:10.10.1.1 Bcast:10.10.1.255 Mask:255.255.255.0
    inet6 addr: 2001:470:...../64 Scope:Global
    inet6 addr: fe80::a60:...../64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:475979 errors:0 dropped:0 overruns:0 frame:0
    TX packets:685155 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:85894169 (81.9 MiB) TX bytes:764015746 (728.6 MiB)


    I Get it! 470 is displayed in a new line when your "ifconfig br0 | awk '/inet/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PIXELIP/" is running - part of the IPv6 address :eek:

    Code:
    10.10.1.250 
    470 
    
     
  6. haarp

    haarp LI Guru Member

    Fixed. I did not expect IPv6 in there.
    Thanks!
     
  7. darkknight93

    darkknight93 Networkin' Nut Member

    :) I have to thank you! Ad-free Internet :>
     
  8. zbeyuz

    zbeyuz Serious Server Member

    I have it work nicely.

    Thank haarp for sharing your wonderful script :D

    By the way, why don't you show us here how to put pixelserv on JFFS or CIFS ?

    I tried to read the thread that you quoted, but I couldn't get it to work.

    Can you write a simple tutorial for that ? To make your thread become the most completed ads-blocking guide for Tomato firmware :)
     
  9. darkknight93

    darkknight93 Networkin' Nut Member

    EDIT: can be ignored due fixed from TE
     
  10. haarp

    haarp LI Guru Member

    Why don't you use the new fixed version?
     
  11. philess

    philess Networkin' Nut Member

    Wow this sounds great! Thanks for sharing!! Will try it out tomorrow, curious if Pixelserv and this can be run on br0 and br1 at the same time... (guests & private network)
     
  12. Ernesto Elias

    Ernesto Elias Serious Server Member

    I cannot get this to work for some reason. I copy and pasted the script in the wan up side of the scripts section and restarted the router just for good measure as I was tweaking another setting. Am I doing something wrong here ?
     
  13. ryzhov_al

    ryzhov_al Networkin' Nut Member

    Well, I've got a solution for that: AdBlock Plus on router.
     
  14. haarp

    haarp LI Guru Member

    Check the router logs! It prints plenty of debug info there. :)

    Nice! That is however beyond the scope of my script. Using Adblock rules would indeed be the ultimate solution, but it involves running a transparent proxy, which is very taxing on the router and not what many people want.

    I also choose to not have it strip the subdomains from all hosts, because that leads to many false positives.

    Can you give me more info? How does running several bridges work, exactly? I might be able to tweak it to work on any number of bridges.

    I'll see what I can do!
     
    zbeyuz likes this.
  15. philess

    philess Networkin' Nut Member

    What would be nice-to-have is to call the script with a parameter ("update") to manually update the lists, instead of on or off for every 24h, so we can set it ourselves easily through scheduler (every monday morning, etc).
     
  16. haarp

    haarp LI Guru Member

    Just set AUTOUPDATE=0 and call the script without arguments. It'll update just fine ;)
     
    philess likes this.
  17. philess

    philess Networkin' Nut Member

    It works fine so far with two instances of pixelserv, one on br0 and one for br1.
    Pixelserv without parameters listens on all IP´s but that would interfere with the
    WebUI on port 80... So i guess there is no way around running one instance per
    interface then. I modified the script just slightly:

    Code:
    #!/bin/sh
    ## Clean, Lean and Mean Adblocking
    ## Tomato WAN Up script v3.3 by haarp
    ##
     
    ### Settings ###
    AUTOUPDATE="0"            ## automatically update every 24h (default: 1)
    PIXELIP="253"            ## last octet of IP to run pixelserv on (1-254)
                    ## or 0 to disable pixelserv (default: 254)
    PIXELBIN="/opt/tomato/adblock/pixelserv"    ## path to pixelserv (v30) binary
    PIXELOPTS=""            ## optional arguments to pass to pixelserv
    GENFILE="/etc/dnsmasq.custom"    ## dnsmasq custom config (must be sourced by dnsmasq!)
     
    ### Sources (uncomment desired blocklists) ###
    ## MVPS HOSTS (~600k) [default]
    SOURCES="$SOURCES http://winhelp2002.mvps.org/hosts.txt"
    ## pgl.yoyo.org (~70k) [default]
    SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"
    ## hosts-file.net (~400k) [optional]
    #SOURCES="$SOURCES http://hosts-file.net/ad_servers.asp"
    ## Hosts File Project (~3000k! needs lots of free mem!) [optional]
    #SOURCES="$SOURCES http://hostsfile.mine.nu/Hosts"
     
    ### Blacklist additional sites (add inside the quotes, space-separated) ###
    BLACKLIST="google-analytics.com"
     
    ### Whitelist sites from blocking (add inside the quotes, space-separated) ###
    WHITELIST="de.ign.com followerscounter.com"
     
     
     
    elog() {
        logger -t ADBLOCK -s "$@"
    }
     
    elog "Download starting"
    until ping -q -c1 google.com >/dev/null; do
        elog "Waiting for internet to come up..."
        sleep 30
    done
     
    echo -n "" > $GENFILE
    for s in $SOURCES; do
        {
            (wget $s -O - || elog "Failed: $s") | \
            tr -d "\r" | \
            sed -e '/^[[:alnum:]:]/!d' | \
            awk '{print $2}' | \
            sed -e '/^localhost$/d' >> $GENFILE
        } &
    done
     
    wait
     
    if [ -s $GENFILE ]; then
        elog "Download finished"
    else
        elog "Failed: Download unsuccessful, aborting"
        rm $GENFILE
        exit 1
    fi
     
    if [ "$PIXELIP" == "0" ]; then
        redirip="0.0.0.0"
        killall $(basename $PIXELBIN) &>/dev/null
        ifconfig br0:1 down &>/dev/null
     
        # added for br1 guest-network
        ifconfig br1:1 down &>/dev/null
     
    else
        redirip=$(ifconfig br0 | awk '/inet addr/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PIXELIP/")
        redirmask=$(ifconfig br0 | awk '/inet addr/{print $4}' | awk -F":" '{print $2}')
        redirbcast=$(ifconfig br0 | awk '/inet addr/{print $3}' | awk -F":" '{print $2}')
        # added for br1 guest-network
        redirip2=$(ifconfig br1 | awk '/inet addr/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PIXELIP/")
        redirmask2=$(ifconfig br1 | awk '/inet addr/{print $4}' | awk -F":" '{print $2}')
        redirbcast2=$(ifconfig br1 | awk '/inet addr/{print $3}' | awk -F":" '{print $2}')
        if ps | grep -v grep | grep -q "$PIXELBIN $redirip"; then
            elog "pixelserv already running, skipping"
        else
            if [ -x $PIXELBIN ]; then
                elog "Setting up pixelserv on $redirip"
                ifconfig br0:1 $redirip netmask $redirmask broadcast $redirbcast up
                # added for br1 guest-network
                elog "Setting up pixelserv on $redirip2"
                ifconfig br1:1 $redirip2 netmask $redirmask2 broadcast $redirbcast2 up
                killall $(basename $PIXELBIN) &>/dev/null
                $PIXELBIN $redirip $PIXELOPTS
                $PIXELBIN $redirip2 $PIXELOPTS
            else
                elog "Failed: $PIXELBIN not found/executable, aborting"
                rm $GENFILE
                exit 2
            fi
        fi
    fi
     
    elog "Generating $GENFILE"
    for b in $BLACKLIST; do
        echo "$b" >> $GENFILE
    done
    for w in $WHITELIST; do
        sed -i -e "/$w/d" $GENFILE
    done
    sort -u $GENFILE -o $GENFILE
    sed -i -e '/^$/d' -e "s:^:address=/:" -e "s:$:/$redirip:" $GENFILE
    # added for br1 guest-network (is this needed tho?)
    sed -i -e '/^$/d' -e "s:^:address=/:" -e "s:$:/$redirip2:" $GENFILE
     
    elog "Config generated, $(wc -l < $GENFILE) unique hosts to block"
     
    elog "Restarting dnsmasq"
    service dnsmasq start
     
    echo "giving dnsmasq some time to start..." >&2
    sleep 15
    elog "Deleting $GENFILE to free memory"
    rm $GENFILE
    # added for br1 guest-network
    elog "Pixelserv should be listening..." >&2
    netstat -l | grep $redirip
    netstat -l | grep $redirip2
     
     
    if [ "$AUTOUPDATE" == "1" ]; then
        [ -x /tmp/script_wanup.sh ] && cru a UpdateAdlist "0 0 * * * /tmp/script_wanup.sh &>/dev/null"
    fi
    Simple "cat /etc/dnsmasq.conf" to check if it is working with "your" dnsmasq version etc.

    Does anyone know of a simple test-site or something to check how it actually acts
    when browsing on a client?
     
  18. haarp

    haarp LI Guru Member

    That may or may not work. The second sed (under "is this needed?") will screw up the config file.

    Thats why I asked if you can tell me more about multi-bridged operation. I can adjust the script to work just fine in this mode.
    What does multi-bridged do? Create two internal networks? Can these networks communicate with each other?
     
  19. philess

    philess Networkin' Nut Member

    Thought so, ok then i´ll remove the second sed in mine.

    Well you could run a check if br1 exists at first, and if so, start pixelserv on both.
    Simply create another bridge "ifconfig br1:1" like i did above and then start pixelserv
    on that one too. Sure these two br0:0 br1:1 could communicate with each other,
    but why should they? I dont think that a route is being generated automatically.
     
  20. haarp

    haarp LI Guru Member

    Because we're using DNS poisoning, that is, redirecting all ad hosts to one server. Two pixelservs is pointless, because you can't resolve a host to two IPs.
    My plan is to redirect them to one pixelserv, but that requires that machines from both bridges can access a server on one bridge. Are you absolutely sure that's possible?
     
  21. philess

    philess Networkin' Nut Member

    Oh ok, now i understand the principle. Yes its definitely possible, but you have to add a route
    from the br1 network to the pixelserv on br0 after creating br1:1. But i dont know the commandline
    for that right now.
     
  22. haarp

    haarp LI Guru Member

    I understand. I could add this route automatically, if somebody shows me how.

    But it'll be strictly optional. An adblock script should not muck with such things without permission.
     
  23. Elfew

    Elfew Addicted to LI Member

    So is possible to add an option to set it only for br0 or br1 network?

    Or just for only some MAC adresses? I think there could be useful the whitelist script from DDwrt and take some parts for mac blocking
     
  24. haarp

    haarp LI Guru Member

    Updated to v3.4, added a selector for the bridge interface. It still only supports a single bridge, though!

    Also added some info on how to install pixelserv to the start post.
     
    Elfew likes this.
  25. Elfew

    Elfew Addicted to LI Member

    Thank you, please can you write me a command for manually update DB? I would like to set auto update to off, and set it in scheduler to specific hour...

    Next thing, I hope that we can make it for more bridges - not only for one... I have one private network on br0 and one public network on br1. So it could be awesome to add this feature in future.

    Another thing, add a whitelist for IP/MAC adresses for which adblovk wont work, just skipped them and allow to see adds etc..
     
  26. haarp

    haarp LI Guru Member

    The command for the scheduler is simply '/tmp/script_wanup.sh'.
    Multiple bridges is currently on the to-do list.
    Exceptions for client machines would introduce massive complexity, and I don't think the DNS-poisoning method can work with it in the first place. I won't add it, sorry.
     
  27. philess

    philess Networkin' Nut Member

    This should work:

    Code:
    ...
     
            if [ -x $PIXELBIN ]; then
                elog "Setting up pixelserv on $redirip"
     
                ifconfig br0:1 $redirip netmask $redirmask broadcast $redirbcast up
           
                # for br1 usage: add route between pixelserv and br1 network
                iptables -I FORWARD -i br0 -p tcp -s $redirip -j ACCEPT
                iptables -I FORWARD -i br1 -p tcp -d $redirip --dport 80 -j ACCEPT
     
                killall $(basename $PIXELBIN) &>/dev/null
                $PIXELBIN $redirip $PIXELOPTS
     
    ...
    Those two iptables lines are the same as when done through the Webui in LAN Access menu,
    should work. But i havent tested it yet properly. Dont have a device at hand that i can easily
    switch over to br1 right now. The rules allow connections FROM pixelserv TO the guestnetwork,
    and FROM the guestnetwork TO pixelserv on port 80.

    Now we could do a grep on the output of ifconfig br1 and check if it exists, and then
    automatically do that iptables route or not. Or simply add a variable at the top and
    let the user set the option himself.
    I think its better not to automatically add the route to br1 because there are sure
    some users who specifically dont want to use adblock on all their networks.
    Best way to do it imho is a option (ENABLEBR1=true) and then add the route,
    if false, only do br0. Oh and some people probably want it the other way around,
    so add a variable for br0 too, so people can easily set on which network they want
    to use it on. (note: when using br1, the iptables rules need to be reversed)

    @Elfew hmm a MAC whitelist would be nice but i dont think thats easily possible with this setup.
    You could add more stuff to the dnsmasq.conf and give different DNS info to some MACs, but
    since all of DNSmasq uses the extended dnsmasq.custom.conf (with all the blocked hosts inside)
    i dont think you can seperate that then. We would need a seperate instance of DNSmasq, one
    which uses the adblock conf, and another one which doesnt. And then give out two different DNS
    ip´s to the clients based on their MAC adress. But i´ll have to look at DNSmasq docu later,
    maybe there is a way to do it with one instance, would be nice to have, but not needed for myself.

    @haarp I would suggest that running the full wanup per scheduler may not be the nice way to go. Could maybe cause problems for people who have other things in their wanup section too. Better to set a variable like "ME=/opt/adblock.sh" and then add that to the scheduler instead of the wanup.sh
     
  28. Elfew

    Elfew Addicted to LI Member

    OK, I just thing that we can easily do that by script from dd-wrt where you can set whitelist/blacklist for interface/or IP range or specific MAC adress and also you can set devices which are not affected by this filter...

    I an send you this script, just PM me if you want... maybe it could help you
     
  29. jerrm

    jerrm Network Guru Member

    Personally, I don't think a script like this should be mucking around in my firewall rules. If it does, then it should be done right - iterate through all bridges, and only when enabled via an optional parameter.
     
    philess likes this.
  30. philess

    philess Networkin' Nut Member

    I would be interested in that script Elfew, but i already expect that most other adblock approaches
    are based on a seperate DNS server to use whitelists. Give me a link or the full script in PM please.
     
  31. philess

    philess Networkin' Nut Member

    Absolutely. I was just pointing out the two rules that would be required (if they work at all haha).
    I am working on the script right now trying to add the options so everyone can set it themselves.

    Edit: Sorry for doublepost, seems i cant delete my own posts :/

    Edit: Ok, this should be good to go:

    Code:
    #!/bin/sh
    ## Clean, Lean and Mean Adblocking
    ## Tomato WAN Up script v3.3 by haarp
    ##
     
    ### Settings ###
    AUTOUPDATE="0"            ## automatically update every 24h (default: 1)
    PIXELIP="253"            ## last octet of IP to run pixelserv on (1-254)
                    ## or 0 to disable pixelserv (default: 254)
    PIXELBIN="/opt/tomato/adblock/pixelserv"    ## path to pixelserv (v30) binary
    PIXELOPTS=""            ## optional arguments to pass to pixelserv
    GENFILE="/etc/dnsmasq.custom"    ## dnsmasq custom config (must be sourced by dnsmasq!)
    INTERFACE=br0            ## which network interface to use for pixelserv, default is br0
    ROUTEBR=false            ## if you have a seperate brX network, this will add firewall
                            ## rules to allow routing between them! (only pixelserv/port80, nothing else)
                            ## examples:
                            ## INTERFACE=br0 and ROUTEBR=true => route between pixelserv/br0 and network/br1
                            ## or INTERFACE=br1 and ROUTEBR=true => route between pixelserv/br1 and network/br0
                            ## or INTERFACE=br0 and ROUTEBR=false => no route, pixelserv works only for br0
    INTERFACE2=br1            ## if you want to use the above ROUTEBR, set this to your secondary network, else ignored
     
    ### Sources (uncomment desired blocklists) ###
    ## MVPS HOSTS (~600k) [default]
    SOURCES="$SOURCES http://winhelp2002.mvps.org/hosts.txt"
    ## pgl.yoyo.org (~70k) [default]
    SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"
    ## hosts-file.net (~400k) [optional]
    #SOURCES="$SOURCES http://hosts-file.net/ad_servers.asp"
    ## Hosts File Project (~3000k! needs lots of free mem!) [optional]
    #SOURCES="$SOURCES http://hostsfile.mine.nu/Hosts"
     
    ### Blacklist additional sites (add inside the quotes, space-separated) ###
    BLACKLIST="google-analytics.com"
     
    ### Whitelist sites from blocking (add inside the quotes, space-separated) ###
    WHITELIST="de.ign.com followerscounter.com"
     
     
     
    elog() {
        logger -t ADBLOCK -s "$@"
    }
     
    elog "Download starting"
    until ping -q -c1 google.com >/dev/null; do
        elog "Waiting for internet to come up..."
        sleep 30
    done
     
    echo -n "" > $GENFILE
    for s in $SOURCES; do
        {
            (wget $s -O - || elog "Failed: $s") | \
            tr -d "\r" | \
            sed -e '/^[[:alnum:]:]/!d' | \
            awk '{print $2}' | \
            sed -e '/^localhost$/d' >> $GENFILE
        } &
    done
     
    wait
     
    if [ -s $GENFILE ]; then
        elog "Download finished"
    else
        elog "Failed: Download unsuccessful, aborting"
        rm $GENFILE
        exit 1
    fi
     
    if [ "$PIXELIP" == "0" ]; then
        redirip="0.0.0.0"
        killall $(basename $PIXELBIN) &>/dev/null
        ifconfig $INTERFACE:1 down &>/dev/null
    else
        redirip=$(ifconfig $INTERFACE | awk '/inet addr/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PIXELIP/")
        redirmask=$(ifconfig $INTERFACE | awk '/inet addr/{print $4}' | awk -F":" '{print $2}')
        redirbcast=$(ifconfig $INTERFACE | awk '/inet addr/{print $3}' | awk -F":" '{print $2}')
        if ps | grep -v grep | grep -q "$PIXELBIN $redirip"; then
            elog "pixelserv already running, skipping"
        else
            if [ -x $PIXELBIN ]; then
                elog "Setting up pixelserv on $redirip"
     
                ifconfig $INTERFACE:1 $redirip netmask $redirmask broadcast $redirbcast up
         
         
                if $ROUTEBR ; then
     
                                # add iptables rules to route between pixelserv and the secondary network
                                elog "Adding iptables rules to enable pixelserv for $INTERFACE2 too"
                                iptables -I FORWARD -i $INTERFACE -p tcp -s $redirip -j ACCEPT
                                iptables -I FORWARD -i $INTERFACE2 -p tcp -d $redirip --dport 80 -j ACCEPT
                                elog "Restarting firewall service..."
                                service firewall restart
     
                fi
         
     
                killall $(basename $PIXELBIN) &>/dev/null
                $PIXELBIN $redirip $PIXELOPTS
            else
                elog "Failed: $PIXELBIN not found/executable, aborting"
                rm $GENFILE
                exit 2
            fi
        fi
    fi
     
    elog "Generating $GENFILE"
    for b in $BLACKLIST; do
        echo "$b" >> $GENFILE
    done
    for w in $WHITELIST; do
        sed -i -e "/$w/d" $GENFILE
    done
    sort -u $GENFILE -o $GENFILE
    sed -i -e '/^$/d' -e "s:^:address=/:" -e "s:$:/$redirip:" $GENFILE
    elog "Config generated, $(wc -l < $GENFILE) unique hosts to block"
     
    elog "Restarting dnsmasq"
    service dnsmasq start
     
    echo "giving dnsmasq some time to start..." >&2
    sleep 15
    elog "Deleting $GENFILE to free memory"
    rm $GENFILE
     
    if [ "$AUTOUPDATE" == "1" ]; then
        [ -x /tmp/script_wanup.sh ] && cru a UpdateAdlist "0 0 * * * /tmp/script_wanup.sh &>/dev/null"
    fi
    I hope the options are well enough explained.

    On a sidenote: All of this will obviously only work if your clients are using the routers DNSmasq as their provider or you enable the "Intercept DNS" option in the WebUI. Clients who can use their own DNS, for example OpenDNS, will still see the ads.
     
  32. Bird333

    Bird333 Network Guru Member

    Harp what specific problems were you trying to solve over the other methods? Can you specify an entire IP for pixelserv?
     
  33. haarp

    haarp LI Guru Member

    That its precisely how I intend it to work.

    I mentioned the reasons in the first post. No, you can't specify an entire IP. The script spawns pixelserv itself, so there is no reason for that. Why do you need this option?


    philess, I'll release an updated version later today.
     
  34. Bird333

    Bird333 Network Guru Member

    I guess I don't need to specify the entire IP. I'll try your script after you add the multi-bridge option. I appreciate your efforts.
     
  35. Bird333

    Bird333 Network Guru Member

    philess- Is this rule really necessary?
    Code:
    iptables -I FORWARD -i br0 -p tcp -s $redirip -j ACCEPT
    Iptables should return a response automatically. Did you try it without that rule?
     
  36. philess

    philess Networkin' Nut Member

    What exactly do you mean? I havent tried it with only the other line. I am pretty sure that
    both lines are required: one for allowing access from pixelserv towards the guest network,
    and the other one for access from guests to pixelserv.
    And if you dont add any iptables rule for this at all, i am very sure that your default
    routing will not work. Because this script creates a new br0:1 for pixelserv (192.168.1.253)
    which is in addition to your standard router ip (.254), tomato has no idea that the new
    .253 now exists and therefore there are no forwarding rules for that ip.
    You can try to run the script without these rules, but i am sure you cannot ping the pixelserv
    ip from the other network then (unless you already have a rule placed yourself that allows
    routing between the two networks).
     
  37. haarp

    haarp LI Guru Member

    Could you guys please test that? My update is almost finished, but I want to know the least iptables rules needed (if any) to make this work. My own setup only uses a single bridge, so I cannot test it.
     
  38. Bird333

    Bird333 Network Guru Member

    What I mean is that once a packet is accepted the return packet(s) are automatically accepted. I think you only need
    Code:
    iptables -I FORWARD -i br1 -p tcp -d $redirip --dport 80 -j ACCEPT
     
  39. philess

    philess Networkin' Nut Member

    Ok i tested without and with those two iptables rules now.

    I dont have many options for devices to switch to my br1 around right now
    so i just used my smartphone and did a ping on the pixelserv ip.
    I suppose if ping works, getting a gif from port 80 should work too.

    Anyhow, it seems the rules are NOT required. Ping always worked on the
    pixelserv ip, regardless of the rules. And now i realized that is likely because
    br0:1 (pixelserv) is directly "bridged" or whatever, to the real br0, therefore
    it is part of the normal interface, and the routing is allowed to it.
    That is good news :) So far.

    It should be noted that users who are using a guest-network, and then use
    a iptables rule to block access for the guests to, for example, SSH on the router,
    they need a rule that blocks SSH on that new pixelserv ip too.
    Or use a rule that simply blocks everything that connects directly to the
    router itself, with exceptions like DNS/DHCP (as discussed in another thread here).
    I myself was able to SSH into the router over the pixelserv ip, while at the same
    time the normal router ip was blocked. Access to the Tomato WebUI was not
    possible tho (probably the most important thing), i think thats because the
    httpd for the WebUI only binds to one ip at start, unlike the SSH which seems
    to bind to all. So anyhow, just a note for those block guests from certain stuff,
    this script with multibridge needs extra rules for blocking on the new ip.
    Or use a rule that blocks everything.
    This has been discussed at length in this thread for example:
    http://www.linksysinfo.org/index.php?threads/restrict-access-to-web-ui-on-guest-wifi.68385/

    And to come to an end, i came across this ad-link with which it is easy to test if
    pixelserv is working or not:

    http://rcm.amazon.com/e/cm?t=thedailydownl-20&o=1&p=11&l=ez&f=ifr&f=ifr

    If its working, you will only get a 1x1 gif, if not, you get a banner.
    When testing on/off pixelserv, make sure to flush your OS´s DNS cache (eg "ipconfig /flushdns")
    and close/open the browser before reloading that link.

    Edit:
    Very good point! But as just posted above, we dont seem to need any rule at all in this case :)
     
  40. haarp

    haarp LI Guru Member

    Oh, whatever. :p Here's the new version for testing:

    Code:
    #!/bin/sh
    ## Clean, Lean and Mean Ad-blocking
    ## Tomato WAN Up script v3.5_pre1 by haarp
    ##
     
    ### [Settings] ###
    PXLBIN="/cifs1/pixelserv"    ## path to pixelserv (v30) binary
    PXLIP="254"            ## last octet of IP to run pixelserv on (1-254)
                    ## or 0 to disable pixelserv (default=254)
    PXLOPTS=""            ## additional options to pass to pixelserv
     
    AUTOUPD="0"            ## auto update every 24h (default=1)
    TARGET="/etc/dnsmasq.custom"    ## dnsmasq custom config (must be sourced by dnsmasq!)
     
    BRIDGE="br0"            ## primary bridge interface for pixelserv (default=br0)
    ROUTEBRIDGES="0"        ## when other bridges are present, add routes to primary
                    ## to allow hosts on them access to pixelserv (default=0)
     
    ## Sources (uncomment desired blocklists) ##
    ## MVPS HOSTS (~600k) [default]
    SOURCES="$SOURCES http://winhelp2002.mvps.org/hosts.txt"
    ## pgl.yoyo.org (~70k) [default]
    SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"
    ## hosts-file.net (~400k) [optional]
    #SOURCES="$SOURCES http://hosts-file.net/ad_servers.asp"
    ## Hosts File Project (~3000k!!) [optional]
    #SOURCES="$SOURCES http://hostsfile.mine.nu/Hosts"
     
    ## Blacklist additional sites (add inside the quotes, space-separated) ##
    BLACKLIST="google-analytics.com"
     
    ## Whitelist sites from blocking (add inside the quotes, space-separated) ##
    WHITELIST="de.ign.com followerscounter.com"
     
    ### [/Settings] ###
     
    alias elog='logger -t ADBLOCK -s'
     
    elog "Download starting"
    until ping -q -c1 google.com >/dev/null; do
        elog "Waiting for internet to come up..."
        sleep 30
    done
     
    echo -n "" > $TARGET
    for s in $SOURCES; do
        {
            (wget $s -O - || elog "Failed: $s") | \
            tr -d "\r" | \
            sed -e '/^[[:alnum:]:]/!d' | \
            awk '{print $2}' | \
            sed -e '/^localhost$/d' >> $TARGET
        } &
    done
     
    wait
     
    if [ -s $TARGET ]; then
        elog "Download finished"
    else
        elog "Failed: Download unsuccessful, aborting"
        rm $TARGET
        exit 1
    fi
     
    pxltables() {
        local i=0
        if [ "$1" == "-I" ]; then
            iptables -L FORWARD | grep -q "tcp dpt:www" && return
        elif [ "$1" == "-D" ]; then
            local redirip=$(iptables -L FORWARD | awk '/tcp dpt:www/{print $5}')
        fi
        while grep -q br$i /proc/net/dev; do
            [ "br$i" == "$BRIDGE" ] && continue
            ##iptables $1 FORWARD -i $BRIDGE -p tcp -s $redirip -j ACCEPT
            iptables $1 FORWARD -i br$i -p tcp -d $redirip --dport 80 -j ACCEPT
            i=$(($i+1))
        done
    }
    if [ "$PXLIP" == "0" ]; then
        elog "pixelserv disabled"
        redirip="0.0.0.0"
        killall $(basename $PXLBIN) &>/dev/null
        pxltables "-D" &>/dev/null
        ifconfig $BRIDGE:1 down &>/dev/null
    else
        redirip=$(ifconfig $BRIDGE | awk '/inet addr/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PXLIP/")
        redirmask=$(ifconfig $BRIDGE | awk '/inet addr/{print $4}' | awk -F":" '{print $2}')
        redirbcast=$(ifconfig $BRIDGE | awk '/inet addr/{print $3}' | awk -F":" '{print $2}')
        if ps | grep -v grep | grep -q "$PXLBIN $redirip"; then
            elog "pixelserv already running, skipping"
        else
            if [ ! -x $PXLBIN ]; then
                elog "Failed: $PXLBIN not found/executable, aborting"
                rm $TARGET
                exit 2
            fi
     
            elog "Setting up pixelserv on $redirip"
     
            ifconfig $BRIDGE:1 $redirip netmask $redirmask broadcast $redirbcast up
            if [ "$ROUTEBRIDGES" == "1" ]; then
                elog "Setting up bridge routes"
                pxltables "-I"
            fi
     
            killall $(basename $PXLBIN) &>/dev/null
            $PXLBIN $redirip $PXLOPTS
        fi
    fi
     
    elog "Generating $TARGET"
    for b in $BLACKLIST; do
        echo "$b" >> $TARGET
    done
    for w in $WHITELIST; do
        sed -i -e "/$w/d" $TARGET
    done
    sort -u $TARGET -o $TARGET
    sed -i -e '/^$/d' -e "s:^:address=/:" -e "s:$:/$redirip:" $TARGET
    elog "Config generated, $(wc -l < $TARGET) unique hosts to block"
     
    elog "Restarting dnsmasq"
    service dnsmasq start
     
    echo "giving dnsmasq some time to start..." >&2
    sleep 15
    elog "Deleting $TARGET to free memory"
    rm $TARGET
     
    [ "$AUTOUPD" == "1" -a -x /tmp/script_wanup.sh ] && cru a UpdateAdlist "0 0 * * * /tmp/script_wanup.sh &>/dev/null"
    If you do things, do them right! ;) Sadly this bloated the script up to 3.9kB. I've commented the second iptables line, go test with and without and let me know


    edit: Are you kidding?!
     
  41. darkknight93

    darkknight93 Networkin' Nut Member

    Isnt it possibe to run 2 dnsmasqs as DNS Server for each br and give them different configs? :S
     
  42. philess

    philess Networkin' Nut Member

    Hmm should definitely be possible. And then use whitelists for MAC addresses etc.
    But... the point of haarp´s script is mostly to be fast and simple i think.
    Running two instances of DNSmasq and two configs etc, that would be a) a lot of work and b) bloat it up.

    One easy "solution" for whitelists would be tho to use the internal DNSmasq for pixelserv (as it is right now),
    and then give out a different DNS by DHCP to your whitelisted clients, for example OpenDNS.
    That could be done with only one line per client in the DNSmasq custom config.

    See here: http://www.linksysinfo.org/index.ph...ntal-controls-with-opendns.68123/#post-222715
     
  43. darkknight93

    darkknight93 Networkin' Nut Member

    Thanks for your Input!
     
  44. Bird333

    Bird333 Network Guru Member

    Actually, we probably need to use INPUT instead of FORWARD since pixelserv is running on the router itself. I'm pretty sure I am going to have to allow the traffic because I have all traffic from br1 blocked going to the router. Given the nature of iptables (needing to be in a certain order) and the fact that everyone is going to have different, extra firewall rules it may be better to just have a note in the script that if you need support on other bridges that you will manually have to alter your firewall rules. It would be nice for the script to do it automatically, but I don't know how possible that really is.
     
    philess likes this.
  45. mstombs

    mstombs Network Guru Member

    See the dd-wrt script mentioned in first post, the how-to-geek guide shows how to configure winscp, which is the easiest way to transfer files from your PC to the router /jffs.

    I haven't tried this script yet, but I see it is growing! One thing that should be considered is that the wan-up script can be run multiple times, especially when making web gui changes and/or a dodgy ISP that keeps dropping/reconnecting - bad things can happen if the script is not protected against be re-entered in another instance!
     
    philess likes this.
  46. philess

    philess Networkin' Nut Member


    True. A note about that should suffice. Those users who already block everything from their guest network,
    with selected exceptions, they probably know how to allow Pixelserv too then.
    Those who dont bother with that, they can start the script and Pixelserv will work on br0 and br1 fine.
    One script alone doesnt have to consider every single scenario possible :) Also, as haarp said before,
    it should be fast and simple. And i like that a lot about his version :)

    Good point! But right now if you start the script again, all it does is download the blocklists again,
    then notice that pixelserv is already running and exits. No risk there.
    But, it should check at first if the script is already active, and if so, skip everything (unless parameter = update).
    Or we could compare the dates of the blocklists before we download them, that would make
    "accidental" starts from wanup even faster, but checking filedates before you actually download them
    is not really reliable i think, if at all possible.

    Maybe INIT is a better place for the script then. Start it only once. And then use the scheduler
    to update the lists every now and then.
     
  47. haarp

    haarp LI Guru Member

    I'll have a look at these things tomorrow. An "update" mode is hard. If I could check the timestamps on the server, it might be possible, but the busybox environment on the firmware does not provide the necessary tools.

    Oh, and it does restart dnsmasq, even if it notices pixserv already running. I have to do this, because I have no way of checking if the dnsmasq that is running runs with or without the blocklist.


    For now, could you guys PLEASE figure out which iptables are definitely needed/not needed now for multi-bridged operation and which iptables rule I can use to prevent access to anything but port 80 on the second IP?
     
    philess likes this.
  48. Bird333

    Bird333 Network Guru Member

    Is this the only place I would need to change the iptables rules in your script for testing purposes?
    Code:
    fi
        while grep -q br$i /proc/net/dev; do
            [ "br$i" == "$BRIDGE" ] && continue
            ##iptables $1 FORWARD -i $BRIDGE -p tcp -s $redirip -j ACCEPT
            iptables $1 FORWARD -i br$i -p tcp -d $redirip --dport 80 -j ACCEPT
            i=$(($i+1))
    Also I assume it is safe to add /usr/sbin/ to the iptables command to eliminate possible issues with the path?

    This should stop everything except connections to port 80. However, this won't be needed by people like me that block everything to the router already.
    Code:
    /usr/sbin/iptables -I INPUT -i br1 -d $redirip --dport !80 -j DROP
     
  49. philess

    philess Networkin' Nut Member

    To work, there are definitely NO required rules with multi-bridged setups.

    But some users would want to restrict access on that new pixelserv ip. Question is tho if that
    should be at all part of the script or not. A note about it and maybe link to one of the threads
    here should suffice.

    Code:
    # this rule blocks everything from guestnetwork to pixelserv ip
    iptables -I INPUT -i br1 -d $redirip -j DROP
    # this rule would allow connections from guestnetwork to pixerlserv ip on port 80 tcp
    iptables -I INPUT -i br1 -p tcp --dport 80 -d $redirip -j ACCEPT
    iptables rules are processed from top to bottom, so the block all rule has to be
    added first (= #1 rule), and then the allow port80 rule (= new #1 rule, other is #2).
    I will test that later today or tomorrow the latest.

    And thank you so much for all your effort haarp! :)
     
  50. jerrm

    jerrm Network Guru Member

    Possible, and pretty reliable unless the server is wonky, but not using busybox wget. BB wget will always stamp the file with the time at download, not the server's "last modified" header.

    The "real" wget actually has a parameter to only download if newer.
     
    philess likes this.
  51. philess

    philess Networkin' Nut Member

    Thanks! Good to know for the future.
     
  52. haarp

    haarp LI Guru Member

    Thanks for those iptables rules. Is there a way to block port!=80 without specifying the bridge? i.e. block for any number of possibly existing/nonexisting bridges by default?

    I have currently reverted the changes in v3.5_pre1 because they bloated the code up quite a lot. Afaik, the WAN up script must stay below 4k, and I was dangerously close to it. Those who do fancy bridged setups can figure out the iptables stuff themselves, but what I want to do is at least block all traffic that is not port 80 by default to the second IP.

    I'll introduce some other changes aswell, tomorrow.
     
  53. philess

    philess Networkin' Nut Member

    Hmm if you want to exclude the bridge from the rule, this should work:

    Code:
    # block everything towards pixelserv
    iptables -I INPUT -d $redirip -j DROP
    # allow tcp port80 towards pixelserv
    iptables -I INPUT -d $redirip -p tcp --dport 80 -j ACCEPT
    Conditions like -d $redirip can be reversed: -d !$redirip to match
    everything that is NOT destinated for pixelserv ip. But that doesnt
    seem to be possible for the --dport option.

    I dont have time to test right now tho, gonna do tomorrow.
     
  54. Bird333

    Bird333 Network Guru Member

    That's cool. As long as the script can work with multiple bridges. :) Keep in mind some may use more than just one additional bridge, they may use two or more (i.e. br0, br1, br2, br3, etc).

    Yes you can just leave the "-i br1" out of the rule. You should be able to use "--dport !80" in the rule. I'll check the syntax when I get home.
     
  55. Bird333

    Bird333 Network Guru Member

    Two problems with the rule. First I needed to specify protocol '-p tcp' in the rule and second I needed a space between the "!" and "80". The rule should look like this.
    Code:
    /usr/sbin/iptables -I INPUT -i br1 -p tcp -d $redirip --dport ! 80 -j DROP
     
  56. Monk E. Boy

    Monk E. Boy Network Guru Member

    FWIW, as was recently described in another thread, by default br1 (probably br2, br3, etc. too but that wasn't discussed) has access to services running on the router on br0. The only way to prevent this access is to create iptables rules (on the INPUT table) to prevent br1 clients from accessing br0's IP address.

    So long as the router's br1 IP address is the default address for br1 clients the router should direct packets to the br0's pixelserv automatically. If you've created iptables rules to prevent br1 from accessing br0, just change the iptables so br1 clients can access pixelserv on br0's IP address.
     
  57. haarp

    haarp LI Guru Member

    3.5 released, changelog in the first post. I've also updated the start post with additional information.

    Multi-bridged operation is now entirely up to the user again, but the bridge for pixelserv to run on can still be selected.

    I have removed some options from ifconfig! It works here, and should work for you. But let me know if you're having trouble, especially if your netmask is not 255.255.255.0

    Setting PIXELIP to 0 does not automagically undo the iptables rules! This rare corner case was deemed too unreliable to include. They shouldn't interfere with normal operation anyway.

    There probably won't ever be an "update" mode. The busybox environment does not give me the tools necessary to check the timestamp of files on a server. Plus, some sources (namely hpHosts) do not send a correct Last-Modified header.
     
  58. Bird333

    Bird333 Network Guru Member

    I just want to be clear. Does this script serve other bridges too or not? I know we have to manually set iptables rules. The answer may be obvious but my brain is a little fried. :)
     
  59. jerrm

    jerrm Network Guru Member

    The
    It can serve any internal IP, you just need to have iptables rules enabling traffic to the IP address pixelserv is bound to. It is the user's responsibility to make sure iptables allows the traffic.
     
  60. philess

    philess Networkin' Nut Member

    By default it works without extra rules required. UNLESS you have rules in place
    that deny access to other bridges (eg. from br1 to br0), you wont have to add
    rules to make it work. Or UNLESS you want to deny a bridge access to pixelserv.

    And jerrm is right, it should be up to the user to set his rules accordingly.


    Ok its getting late, i should get off here. I just finished modifying haarp´s script
    a bit. Added update mode (start as 'adblock.sh update') and moved some stuff
    around. But when doing a update of the lists, a restart of DNSmasq is required.
    So pick a time for the auto-updates that is the least inconvient, i changed it
    to 5am on every monday. (i dont think daily updates are necessary). You can
    change the line "cru a UpdateAdlist..." to whatever other time/day you prefer.
    Starting the script for the first time, it will update the rules automatically.
    After then, it will only check if Pixelserv is running and start it if required and
    of course create the required brX:1.

    Summary: start once = start pixelserv & update
    After that: scheduler or manual update: "adblock.sh update"
    Starting it again without parameter will not do anything unless
    Pixelserv is not running. That makes it "safe" to put in WANUP.
    Tho i think INIT is still better, with a "sleep 30" or w/e in front.

    Note: it still gives me a error "[: update: unknown operand"
    when starting the first time. I cant fix that right now.
    It has to do with evaluation the parameter. BUT it works anyway.
    Just the output isnt proper. Probably a tiny mistake, but i need sleep now.

    Code:
    #!/bin/sh
    ## Clean, Lean and Mean Ad-blocking
    ## Tomato WAN Up script v3.5
    ##
    ## by 'haarp' @ linksysinfo.org
    ##
     
    ### Settings ###
    MYSELF="/opt/adblock.sh" ## full path to this script
    AUTOUPDATE="1"            ## auto-update sources every monday @ 5am
    TARGET="/etc/dnsmasq.custom"    ## dnsmasq custom config (must be sourced by dnsmasq!)
    PIXELIP="254"            ## 0 to disable pixelserv or last octet of IP
                    ## (1-254) to run pixelserv on (default: 254)
    PIXELBIN="/opt/pixelserv"    ## path to pixelserv (v30) binary
    PIXELOPTS=""            ## additional options to pass to pixelserv
    BRIDGE="br0"            ## bridge interface for pixelserv (default: br0)
     
    ### Sources (uncomment desired blocklists) ###
    ## MVPS HOSTS (~600k) [default]
    SOURCES="$SOURCES http://winhelp2002.mvps.org/hosts.txt"
    ## pgl.yoyo.org (~70k) [default]
    SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"
    ## hpHosts ad/tracking servers (~400k) [optional]
    #SOURCES="$SOURCES http://hosts-file.net/ad_servers.asp"
    ## Hosts File Project (~3M!!) [optional]
    #SOURCES="$SOURCES http://hostsfile.mine.nu/Hosts"
    ## MalwareDomainList.com (~40k) [optional]
    SOURCES="$SOURCES http://www.malwaredomainlist.com/hostslist/hosts.txt"
     
    ### Blacklist additional sites (add inside the quotes, space-separated) ###
    BLACKLIST="google-analytics.com"
     
    ### Whitelist sites from blocking (add inside the quotes, space-separated) ###
    WHITELIST="de.ign.com followerscounter.com"
     
     
    alias elog='logger -t ADBLOCK -s'
    alias iptables='/usr/sbin/iptables'
    doupdate=false
    startup=true
    exitupdate=false
     
    sighandler() {
        elog "Fail: Execution aborted"
        rm $TARGET &>/dev/null
        rm $pidfile &>/dev/null
        exit 100
    }
    trap sighandler SIGQUIT SIGINT SIGTERM SIGHUP
     
     
    if [ ! -f adblock.chk ] ; then
        elog "First time starting, will update sources"
            if [ "$AUTOUPDATE" == "1" -a -x $MYSELF ]; then
                cru a UpdateAdlist "0 5 1  * * $MYSELF update &>/dev/null"
                elog "Auto-update added to cronjobs"
            else
                cru d UpdateAdlist
            fi
        doupdate=true
        touch adblock.chk
    fi
     
    if [ $1 = "update" ] ; then
        doupdate=true
        exitupdate=true
        elog "Update mode"
    fi
     
    if $doupdate ; then
     
            elog "Download starting"
            until ping -q -c1 google.com >/dev/null; do
                elog "Waiting for internet to come up..."
                sleep 30
            done
           
            echo -n "" > $TARGET
            for s in $SOURCES; do
                {
                    (wget $s -O - || elog "Failed: $s") | \
                    tr -d "\r" | \
                    sed -e '/^[[:alnum:]:]/!d' | \
                    awk '{print $2}' | \
                    sed -e '/^localhost$/d' >> $TARGET
                } &
            done
           
            wait
           
            if [ -s $TARGET ]; then
                elog "Download finished"
            else
                elog "Fail: Download unsuccessful, aborting"
                rm $TARGET
                exit 2
            fi
     
            elog "Generating $TARGET"
            for b in $BLACKLIST; do
                echo "$b" >> $TARGET
            done
            for w in $WHITELIST; do
                sed -i -e "/$w/d" $TARGET
            done
            sort -u $TARGET -o $TARGET
            sed -i -e '/^$/d' -e "s:^:address=/:" -e "s:$:/$redirip:" $TARGET
            elog "Config generated, $(wc -l < $TARGET) unique hosts to block"
     
            elog "Restarting dnsmasq"
            service dnsmasq start
     
            sleep 10
            elog "All done, removing $TARGET to free memory"
            rm $TARGET;
            if $exitupdate; then
                    exit 0
            fi
     
    fi
     
     
    pidfile=/var/run/adblock.pid
    if [ -e $pidfile ]; then
        elog "Fail: Another instance found ($pidfile exists), aborting"
        exit 10
    fi
    echo $$ > $pidfile
     
     
    if [ "$PIXELIP" == "0" ]; then
        elog "pixelserv disabled"
        killall $(basename $PIXELBIN) &>/dev/null
        ifconfig $BRIDGE:1 down &>/dev/null
        redirip="0.0.0.0"
    else
        redirip=$(ifconfig $BRIDGE | awk '/inet addr/{print $3}' | awk -F":" '{print $2}' | sed -e "s/255/$PIXELIP/")
        if ps | grep -v grep | grep -q "$PIXELBIN $redirip"; then
            elog "pixelserv already running, skipping"
        else
            if [ ! -x $PIXELBIN ]; then
                elog "Fail: $PIXELBIN not found or not executable, aborting"
                rm $TARGET
                exit 3
            fi
           
     
            elog "Setting up pixelserv on $redirip"
     
            ifconfig $BRIDGE:1 $redirip up
            iptables -vL INPUT | grep $BRIDGE | grep -q "$redirip *tcp dpt:www" || {
                iptables -I INPUT -i $BRIDGE -p all -d $redirip -j DROP
                iptables -I INPUT -i $BRIDGE -p tcp -d $redirip --dport 80 -j ACCEPT
            }
     
            killall $(basename $PIXELBIN) &>/dev/null
            $PIXELBIN $redirip $PIXELOPTS
        fi
    fi
     
    rm $pidfile; exit 0
     
  61. haarp

    haarp LI Guru Member

    I do not understand what you're trying to do here. What's "update" to you? If pixelserv is running, the script only fetches new sources and restarts dnsmasq, the latter of which barely takes more than maybe a second, even on a WRT54G. Sure, you lose your DNS cache, but that's about it (it gives me an idea tho on how to avoid that, though). Unless we can get timestamps from the blocklist sources, an update function will not be very beneficial.

    I couldn't find the bug from my quick skim, but I see that you dumped the pidfile stuff further down. That's needed for detection of multiple instances and should be further up top. You also used a slightly outdated version of 3.5 as your base. Sorry, I made some last-minute changes without incrementing the version :p

    I'll implement the changes to avoid dnsmasq restarts when nothing changed tomorrow.
     
  62. philess

    philess Networkin' Nut Member

    I realized later on that the DNSmasq restart is required after updating the sources, like i mentioned above.
    Still someone here seemed eager to have a update function haha, well there it is hehe.

    I highly doubt that someone could be running multiple instances of the script, all with the update
    parameter. Imho the pidfile check is a good thing, but since script is not looping itself 24/7 the chances
    that it could be started twice are pretty slim. And also, even if started twice, nothing really bad could
    happen besides some error outputs.

    You could keep the downloaded sources on /opt or /cifs and after downloading a new one, compare them,
    and only if different apply them and restart DNSmasq.
     
  63. zbeyuz

    zbeyuz Serious Server Member

    Does anyone has a problem with Yahoo or Yahoo Mail when the source Pgl.yoyo.org is enabled ?

    It blocks all the ads and also many buttons in the Yahoo Mail UI... I can not even see a trash or a forward button...

    What is wrong ?
     
  64. philess

    philess Networkin' Nut Member

    Try this: Disable the adblock again, go to your Yahoo mail, check the urls that those buttons use,
    add that domain to the whitelist in the adblock, start adblock again. Check.
     
  65. zbeyuz

    zbeyuz Serious Server Member

    Hi philess,

    How to check the urls that those buttons use ?
     
  66. philess

    philess Networkin' Nut Member

    Whatever browser you use, look for a option 'View Source', inspect.
     
  67. haarp

    haarp LI Guru Member

    I figured out a way to check the timestamps on the server using netcat! :D

    Script has been updated. v3.6 will now only do an update when a list is outdated.
    blocklists on https servers or servers using non-standard ports (not 80) will not be able to use this feature, though.

    The script is now about 3.9kB in size. I had to employ a couple of tricks to keep the size in check. Does anyone know if 4k is still the limit for Tomato scripts?
     
    philess likes this.
  68. zbeyuz

    zbeyuz Serious Server Member

    With a new version 3.6. I got an error of "Wan Script is too long. Maximum allowed is 4096 bytes".

    How to increase a limit of tomato firmware. I am using Toastman Build on E4200...
     
  69. Elfew

    Elfew Addicted to LI Member

    Is possible to allow run bigger scripts than 4096 bytes in Tomato? Is kernel related or just set to 4096 bytes?
     
  70. haarp

    haarp LI Guru Member

    It's 3914 bytes. That's definitely below 4096 :| You probably added stuff to the white/blacklist or additional sources, zbeyuz. I'll see what I can do to reduce it further.

    Elfew, it seems to be a limitation in the GUI. The kernel and sh would have no problems handling scripts megabytes in size.
     
  71. Elfew

    Elfew Addicted to LI Member

    OK, so contact Victek/Shibby/Toastman, maybe they can increase it easily...
     
  72. jerrm

    jerrm Network Guru Member

    It's a limitation of the GUI. Not sure if it's just a generic sanity/safety check, or if 4K is the largest variable that can be saved using nvram.
     
  73. haarp

    haarp LI Guru Member

    Ok, I split the blocklists, see the start post. It should be smaller now. You guys either added to the white/blacklists, or Windows included its awful CR chars to each newline :/

    It wouldn't be too hard to make a decoder like the ALL-U-NEED script and avoid hitting the limit, but I want to keep the thing simple and easily readable.
     
  74. Bird333

    Bird333 Network Guru Member

    Is it possible that some invisible characters are being added with the copy and paste? I got that error with the test script a few posts up.
     
  75. RMerlin

    RMerlin Network Guru Member

    There are various stages at which this can cause the size to be limited. There's a buffer size limitation related to the data exchange between your browser and the web server, for instance.

    My advice: put the big script in JFFS, and only tell Tomato to run that script instead of giving Tomato the whole script.
     
    philess likes this.
  76. Frequenzy

    Frequenzy Networkin' Nut Member

    how do we restart the script manually
     
  77. haarp

    haarp LI Guru Member

    v3.6.1 release, added "force" option

    Good question, I added it to the start post:

     
  78. haarp

    haarp LI Guru Member

    v3.7, added bigmem mode, see startpost for more info. This should make restarting the script, except for initial setup after router restart or updating filter lists, unnecessary.
     
    nobugme likes this.
  79. fubdap

    fubdap Addicted to LI Member

    @haarp - great job on this project. I have couple questions.
    (1) I have my pixelserv on a usb stick. so all I have to do on the script is change this:
    PIXELBIN="/cifs1/pixelserv" to point to the usb stick?

    (2) Also I have a handful of white list and I may have more in the future. Will I run the risk of running out of memory, since we are approaching the 4096 bytes limit? Thanks.
     
  80. haarp

    haarp LI Guru Member

    1) Pretty much! Make sure the path is correct, or the script will complain
    2) All I can say is to try it. I'm actually curious what the actual limit is.
     
  81. Bird333

    Bird333 Network Guru Member

    How do you manually stop it if necessary? Where is the config file saved with 'BIGMEM' set?
     
  82. Frequenzy

    Frequenzy Networkin' Nut Member

    just tried and its working.
     
  83. fubdap

    fubdap Addicted to LI Member

    I installed this on my N16. These are the last three lines of my log. What does that mean. It is working?

    Code:
    Apr 26 20:48:53 HiTech user.notice ADBLOCK: Removing /etc/dnsmasq.custom to free memory
    Apr 26 20:48:53 HiTech user.notice ADBLOCK: Done
    Apr 26 20:49:03 HiTech cron.err crond[534]: time disparity of 22783788 minutes detected
    
     
  84. Frequenzy

    Frequenzy Networkin' Nut Member

    yup that shoud work.
     
  85. philess

    philess Networkin' Nut Member

    There is no actual "config" file. To disable/remove the script, make sure /etc/dnsmasq.custom does not exit (just delete it: rm -rf etc/dnsmasq.custom) and then kill pixelserv (killall pixelserv), (service dnsmasq restart). Thats it, you are back to your normal DNS config.
     
  86. zbeyuz

    zbeyuz Serious Server Member

    After 24 hours, the script checks for updating. Here is what I get from the logs:

    Apr 27 00:00:01 RT-C0C1C08823A0 user.notice ADBLOCK: Download Starting
    Apr 27 00:00:03 RT-C0C1C08823A0 user.notice ADBLOCK: Unchanged: http://sysctl.org/cameleon/hosts (Last-Modified: Tue, 23 Apr 2013 03:44:36 GMT)
    Apr 27 00:00:04 RT-C0C1C08823A0 user.notice ADBLOCK: Unchanged: http://winhelp2002.mvps.org/hosts.txt (Last-Modified: Tue, 02 Apr 2013 18:04:16 GMT)
    Apr 27 00:00:04 RT-C0C1C08823A0 user.notice ADBLOCK: Unchanged: http://www.malwaredomainlist.com/hostslist/hosts.txt (Last-Modified: Thu, 25 Apr 2013 06:33:39 GMT)
    Apr 27 00:00:04 RT-C0C1C08823A0 user.notice ADBLOCK: Unchanged: http://hosts-file.net/ad_servers.asp (Last-Modified: Thu, 25 Apr 2013 0:9:5 GMT)
    Apr 27 00:00:04 RT-C0C1C08823A0 user.notice ADBLOCK: Unchanged: http://adaway.sufficientlysecure.org/hosts.txt (Last-Modified: Fri, 29 Mar 2013 16:44:11 GMT)
    Apr 27 00:00:05 RT-C0C1C08823A0 user.notice ADBLOCK: Unchanged: http://someonewhocares.org/hosts/hosts (Last-Modified: Tue, 09 Apr 2013 16:48:11 GMT)
    Apr 27 00:00:05 RT-C0C1C08823A0 user.notice ADBLOCK: Download Finished
    Apr 27 00:00:05 RT-C0C1C08823A0 user.notice ADBLOCK: Pixelserv Already Running, Skipping
    Apr 27 00:00:05 RT-C0C1C08823A0 user.notice ADBLOCK: Generating /etc/dnsmasq.custom
    Apr 27 00:00:05 RT-C0C1C08823A0 user.notice ADBLOCK: Config Generated, 1 Unique Hosts To Block
    Apr 27 00:00:05 RT-C0C1C08823A0 user.notice ADBLOCK: Restarting Dnsmasq


    Why it is only 1 host to block ?
     
  87. haarp

    haarp LI Guru Member

    That shouldn't happen indeed. Can you do cat /etc/dnsmasq.custom under Tools>System and tell me what it returns?


    I'll add a stop function later today. The config is defined as TARGET in the header.
     
  88. jerrm

    jerrm Network Guru Member

    Please add an LMPATH variable to specify where to store the lastmod files.
    With that those that have usb storage can set:
    Code:
    TARGET="/opt/etc/dnsmasq.custom"
    LMPATH="/opt/var/lib/misc"
    Reduce our GUI script to something like:
    Code:
    ln -s /opt/etc/dnsmasq.custom /etc/dnsmasq.custom
    /opt/bin/adblock.sh
    And have the list and lastmod data survive a reboot.
     
  89. zbeyuz

    zbeyuz Serious Server Member


    Here is what I get

    cat: can't open '/etc/dnsmasq.custom': No such file or directory

    I don't know why but the router still able to block ads to Pixelserv.

    I am running Toastman lastest beta builld on E4200 v1.
     
  90. zbeyuz

    zbeyuz Serious Server Member



    Here is a log with version 3.5:
    Apr 27 13:54:37 RT-C0C1C08823A0 user.notice ADBLOCK: Download Is Starting
    Apr 27 13:55:07 RT-C0C1C08823A0 user.notice ADBLOCK: Download Finished
    Apr 27 13:55:07 RT-C0C1C08823A0 user.notice ADBLOCK: Setting Up Pixelserv On 192.168.1.254
    Apr 27 13:55:07 RT-C0C1C08823A0 daemon.info pixelserv[1128]: /jffs/pixelserv V30 compiled: Nov 6 2012 09:13:44 from pixelserv30.c
    Apr 27 13:55:07 RT-C0C1C08823A0 daemon.notice pixelserv[1130]: Listening on 192.168.1.254:80
    Apr 27 13:55:07 RT-C0C1C08823A0 user.notice ADBLOCK: Generating /etc/dnsmasq.custom

    Apr 27 13:56:06 RT-C0C1C08823A0 user.notice ADBLOCK: Config Generated, 116056 Unique Hosts To Block
    Apr 27 13:56:06 RT-C0C1C08823A0 user.notice ADBLOCK: Restarting Dnsmasq
    Apr 27 13:56:19 RT-C0C1C08823A0 user.notice ADBLOCK: All Done, Removing /etc/dnsmasq.custom To Free Memory
    Apr 27 14:00:02 RT-C0C1C08823A0 syslog.info root: -- MARK --
     
  91. jerrm

    jerrm Network Guru Member

    I took a quick glance at the source and the nvram utility executable has a max file size defined as 4096. In the source for the utility, this looks to be a rather arbitrary setting and could probably be changed, but I haven't really looked at all the underlying system level nvram code to see if there is a more significant reason for the restriction. I doubt the maintainers would be eager to change such a core utility regardless.

    Optionally reading white/black lists from a file might be a nice feature. Those with USB storage would not have to touch the script to tweak the lists, and even those without could stretch things a bit by adding the lists to the init section, something like:
    Code:
    cat << EOF > /etc/adblock.white
    domain1.com
    domain2.com
    EOF
     
  92. mstombs

    mstombs Network Guru Member

    If you are going to store pixelserv on jffs - why not store the adblock script there too and just put a link to it in wan-up script? I know there are routers/ firmwares with too small flash to support jffs, but you could store a compressed version of script using gzip and tomato function to store files in nvram "nvram setfile2ram" (auto extracted on boot).

    A example script that applies a whitelist from file is as follows:-

    Code:
    root@wrt54gs:/tmp/home/root# cat /jffs/getadblock.sh
    #!/bin/sh -x
    CNF=/jffs/adblock.conf
    WHL=/jffs/whitelist
    ADS="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext"
    AIP="192.168.0.1" # changes 127.0.0.1 to this globally
    wget -O - "$ADS" | sed "s/127.0.0.1/$AIP/" | grep -Fvf $WHL >$CNF
    
     
    ryzhov_al and zbeyuz like this.
  93. jerrm

    jerrm Network Guru Member

    There is nothing preventing the user from doing that now, but I think haarp wants to try and keep it cut-and-paste simple for those without, and there is the 4K limit to consider.

    Personally, I'd lose the iptables rules and the autoupdate sections in favor of white/blacklist files. I don't really have anything against the rules in the script, but generally don't like things mucking with my rules. The autoupdate has issues if the script is not contained in or called from wanup, or if there are other wanup processes that should not be repeated.
     
  94. haarp

    haarp LI Guru Member

    Yeah, I'm redesigning it from the ground up now.
    Since pixelserv needs some kind of static storage anyway, I might aswell save the script at the same place.
    (I really wish these forums would allow random wget access to attachments btw. I could have the script grab it automatically...)
    That means Wan-Up will only call it, the size limitation disappears and it'll be more flexible. But at the same time it will be slightly more cumbersome to manage.

    I'm currently trying to figure out a nice way to handle settings and lists. If I keep them in the Wan-Up script, that makes the script dependent on the Wan-Up script, killing flexibility. On the other hand, it makes them very easy to modify, which is a goal for me.
    Storing these settings in extra files seems like a good idea, but how does the average admin edit those? Preferably without needing to leave the Tomato GUI? Tricky.

    I'm open for ideas. Maybe I could have a small section in the WAN Up script update the config files. It wouldn't directly interfere with Adblock operation, but still be able to edit settings this way...


    (speaking of scripts, why does Tomato still not have a Wan-Down script section? or an upload file function! that'd be useful!)
     
  95. jerrm

    jerrm Network Guru Member

    For anything related to basic connectivity and security I try to keep it in the GUI. "Extra" features can often live entirely on USB/JFFS, but even then I shy away from the various autorun files and prefer to initiate everything from the GUI scripts. Essentially I want one place to see what is going on with the router.

    Maintaining config files strictly on disk is fine. If there is something I feel needs to be accessible or is easier to do without ssh, I do use the heredocs approach I listed a couple of posts ago.
     
  96. darkknight93

    darkknight93 Networkin' Nut Member

    i really appreciate your work - Things are improving every time i open up this thread :)
     
  97. zbeyuz

    zbeyuz Serious Server Member

    Why do I always get an error after the script is updated, even with version 3.7 ?

    Apr 28 00:00:01 RT-586D8F3E885D user.notice ADBLOCK: Download starting
    Apr 28 00:00:03 RT-586D8F3E885D user.notice ADBLOCK: Unchanged: http://www.malwaredomainlist.com/hostslist/hosts.txt (Last-Modified: Thu, 25 Apr 2013 06:33:39 GMT)
    Apr 28 00:00:03 RT-586D8F3E885D user.notice ADBLOCK: Unchanged: http://hosts-file.net/ad_servers.asp (Last-Modified: Sat, 27 Apr 2013 17:26:13 GMT)
    Apr 28 00:00:03 RT-586D8F3E885D user.notice ADBLOCK: Unchanged: http://winhelp2002.mvps.org/hosts.txt (Last-Modified: Tue, 02 Apr 2013 18:04:16 GMT)
    Apr 28 00:00:04 RT-586D8F3E885D user.notice ADBLOCK: Unchanged: http://adaway.sufficientlysecure.org/hosts.txt (Last-Modified: Fri, 29 Mar 2013 16:44:11 GMT)
    Apr 28 00:00:04 RT-586D8F3E885D user.notice ADBLOCK: Download finished
    Apr 28 00:00:04 RT-586D8F3E885D user.notice ADBLOCK: pixelserv already running, skipping
    Apr 28 00:00:04 RT-586D8F3E885D user.notice ADBLOCK: Generating /etc/dnsmasq.custom
    Apr 28 00:00:04 RT-586D8F3E885D user.notice ADBLOCK: Config generated, 0 unique hosts to block
    Apr 28 00:00:04 RT-586D8F3E885D user.notice ADBLOCK: Restarting dnsmasq

    P/s: I am sorry but I don't understand this line in your instruction:
    • Make sure your firmware features under Advanced->DHCP/DNS this line: "Note: The file /etc/dnsmasq.custom is also added to the end of Dnsmasq's configuration file if it exists."
    Do I need to add anything to Dnsmasq - Custom configuration ?
     
  98. darkknight93

    darkknight93 Networkin' Nut Member

    no you dont Need to add/insert anything in dnsmasq.custom - the script will generate neccessary files itself.

    This line just tells you, that dnsmasq will use the script-generated file for his own configuration - if he doesn't and this line does not exist in your Info-table, adblockscript will not work
     
    zbeyuz likes this.
  99. haarp

    haarp LI Guru Member

    Welp, I just noticed that the logic behind the last-modified check is broken. When one source updates, the blocklist will only contain the newly updated source. Whoops.

    Fixing that brings me to a question, though. The new version will depend on non-volatile storage (JFFS, CIFS, SDcard, USB, whatever). I still intend to keep space usage at a minimum, but JFFS users might run into trouble regardless, depending on the size of their blocklists. There are two way I can handle this now:

    a) Keep each source around separately in addition to the actual blocklist. Requires 2x the space of all blocklists combined (even 3x temporarily), but only redownloads sources that actually changed.
    b) Don't keep individual sources. Requires only the space of all blocklists combined (2x temporarily), but has to redownload ALL sources when any ONE source changes.

    Personally I prefer b. But please let me know which one you'd prefer.


    What darkknight said. Since you're using 3.7 now, I'd like you to enable bigmem mode, rerun the script, then do this command again and tell me what it returns:
    cat /etc/dnsmasq.custom
    If you get no output, then do this please:
    cat /etc/dnsmasq.custom | openssl enc -e -base64
     
  100. zbeyuz

    zbeyuz Serious Server Member


    Personally, I like an A idea. It will give us more freedom to expand our script and ability to add more ads-blocking sources. Especially, when we own a router model like Asus RT-N66U with a lot of RAM.

    I have been enabled Bigmem mode. But when I am executing both scripts, it gives me no results, but errors.
     

Share This Page