1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Alter ad blocking list after download?

Discussion in 'Tomato Firmware' started by Jedis, Oct 23, 2009.

  1. Jedis

    Jedis LI Guru Member

    Hello,

    I was wondering if I could get some help with an ad-blocking script. (specifically rhester72's script here.)

    Code:
    abc=dnsmasq_adblock.conf;tip=192.168.1.150;wget -q -O /tmp/$abc 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext';[ $? -eq 0 -a `grep ^address= /tmp/$abc|wc -l` -gt 0 ] && (logger -t adblock -p 5 Server download OK;cat /tmp/$abc|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ ! -s /cifs1 ] && mv -f /tmp/$abc /cifs1/$abc.bak || rm /tmp/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart) || (logger -t adblock -p 4 Server download failed;[ ! -s /etc/$abc -a -s /cifs1/$abc.bak ] && (logger -t adblock -p 5 Data recovered from backup;cat /cifs1/$abc.bak|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart));unset abc tip
    Unfortunately, there's a few domains blocked by the list that it downloads that I need to unblock. Is there anyway I can add something in the script (before the dnsmasq restart) to have it remove a few domain names from the list and then restart dnsmasq so the changes take effect? I imagine it might involve using sed, but have no clue how to do it.

    Specifically, I need to unblock www.google-analytics.com and ssl.google-analytics.com in particular. Apparently, a game uses them in their online store and a family member is not able to use that part of the game since it is blocked by this script.

    Thanks for any help you can provide!
     
  2. mstombs

    mstombs Network Guru Member

    kinda hard to read that script! Are there really 2 places 127.0.0.1 is replaced?

    My manually updated conf generator which made up from bits on this forum is just

    Code:
    # cat /jffs/getadblock.sh
    #!/bin/sh -x
    CNF=/jffs/adblock.conf
    WHL=/jffs/whitelist
    ADS="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext"
    AIP="192.168.0.1" # changes 127.0.0.1 to this globally
    wget -O - "$ADS"|sed "s/127.0.0.1/$AIP/" | grep -Fvf $WHL >$CNF
    which allows a whitelist, for example

    Code:
    # cat /jffs/whitelist
    edgefcs.net
    #
     
  3. TexasFlood

    TexasFlood Network Guru Member

    What I did for my customized ad blocking script was use grep & a temp file.
    To do it this way, put the entries in a file, like so:
    Code:
    echo "www.google-analytics.com">/tmp/allowlist
    echo "ssl.google-analytics.com">>/tmp/allowlist
    ...etc as many as you have.
    You'd probably want to do this on the init tab then the file is there when you need it.
    Then right after the wget of /tmp/dnsmasq_adblock.conf, you could insert something like the following:
    Code:
    [ -f /tmp/allowlist ]&& (cat /tmp/dnsmasq_adblock.conf | grep -v -f /tmp/allowlist>>/tmp/adblock.tmp; mv -f /tmp/adblock.tmp /tmp/dnsmasq_adblock.conf)
    You could be a little more efficient by creating variables for those filenames.
     
  4. Jedis

    Jedis LI Guru Member

    Do I need to put the entire line in there? The line from the downloaded list is actually
    Code:
    address=/google-analytics.com/127.0.0.1
     
  5. TexasFlood

    TexasFlood Network Guru Member

    No, "grep -v" will omit all lines that match the pattern anywhere in the line. In fact, now that I think about it, for this only one line with "google-analytics.com" or even just "google-analytics" should be needed. If you don't have any more sites, then you shouldn't even need the allow list file and could just do something like:
    Code:
    cat /tmp/dnsmasq_adblock.conf | grep -v "google-analytics">/tmp/adblock.tmp; mv -f /tmp/adblock.tmp /tmp/dnsmasq_adblock.conf
     
  6. Jedis

    Jedis LI Guru Member

    Ok, I'm getting an error when I try to execute the script. Here is what I have:
    Code:
    xyz=allowlist;hij=adblock.tmp;abc=dnsmasq_adblock.conf;tip=192.168.1.1;wget -q -O /tmp/$abc 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext';[ -f /tmp/$xyz ]&&cat /tmp/$abc | grep -v -f /tmp/$xyz>>/tmp/$hij||mv -f /tmp/$hij /tmp/$abc;[ $? -eq 0 -a `grep ^address= /tmp/$abc|wc -l` -gt 0 ] && (logger -t adblock -p 5 Server download OK;cat /tmp/$abc|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ ! -s /cifs1 ] && mv -f /tmp/$abc /cifs1/$abc.bak || rm /tmp/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart) || (logger -t adblock -p 4 Server download failed;[ ! -s /etc/$abc -a -s /cifs1/$abc.bak ] && (logger -t adblock -p 5 Data recovered from backup;cat /cifs1/$abc.bak|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart));unset xyz hij abc tip
    The error is "mv: cannot rename '/tmp/adblock.tmp': No such file or directory"

    Any ideas?
     
  7. TexasFlood

    TexasFlood Network Guru Member

    Try
    Code:
    xyz=allowlist;hij=adblock.tmp;abc=dnsmasq_adblock.conf;tip=192.168.1.1;wget -q -O /tmp/$abc 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext';[ -f /tmp/$xyz ] && (cat /tmp/$abc | grep -v -f /tmp/$xyz>>/tmp/$hij; mv -f /tmp/$hij /tmp/$abc);[ $? -eq 0 -a `grep ^address= /tmp/$abc|wc -l` -gt 0 ] && (logger -t adblock -p 5 Server download OK;cat /tmp/$abc|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ ! -s /cifs1 ] && mv -f /tmp/$abc /cifs1/$abc.bak || rm /tmp/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart) || (logger -t adblock -p 4 Server download failed;[ ! -s /etc/$abc -a -s /cifs1/$abc.bak ] && (logger -t adblock -p 5 Data recovered from backup;cat /cifs1/$abc.bak|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart));unset xyz hij abc tip
    I made at least one typo earlier, think I fixed it, but apparently not before you referenced it. Also, did you create the /tmp/allowlist file in the init tab? Or if you decided that you don't need the file, try
    Code:
    hij=adblock.tmp;abc=dnsmasq_adblock.conf;tip=192.168.1.1;wget -q -O /tmp/$abc 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext';cat /tmp/$abc | grep -v "google-analytics">/tmp/$hij; mv -f /tmp/$hij /tmp/$abc);[ $? -eq 0 -a `grep ^address= /tmp/$abc|wc -l` -gt 0 ] && (logger -t adblock -p 5 Server download OK;cat /tmp/$abc|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ ! -s /cifs1 ] && mv -f /tmp/$abc /cifs1/$abc.bak || rm /tmp/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart) || (logger -t adblock -p 4 Server download failed;[ ! -s /etc/$abc -a -s /cifs1/$abc.bak ] && (logger -t adblock -p 5 Data recovered from backup;cat /cifs1/$abc.bak|sed 's/127.0.0.1/'$tip'/g'>/etc/$abc;[ -h /etc/dnsmasq.custom ] && service dnsmasq restart));unset xyz hij abc tip
     
  8. Jedis

    Jedis LI Guru Member

    Yeah my Init script is as follows:
    Code:
    echo "google-analytics">/tmp/allowlist
    echo "ssl.google-analytics.com">>/tmp/allowlist
    [ ! -f /tmp/dnsmasq.chk ] && (ln -s /etc/dnsmasq_adblock.conf /etc/dnsmasq.custom;touch /tmp/dnsmasq.chk)
    sleep 5
    ip neigh change 192.168.1.254 lladdr ff:ff:ff:ff:ff:ff nud permanent dev br0
    ip neigh add 192.168.1.254 lladdr ff:ff:ff:ff:ff:ff nud permanent dev br0
    I kept the second line in there just so I can easily edit it in the future (so I don't forget the extra > character). I'm not getting the error anymore but haven't had a chance to see if the Google ads stuff is unblocked (I'm working on the router remotely).

    EDIT: Looks like it should be working theoretically:
    Code:
    # cat /etc/dnsmasq_adblock.conf | grep google
    address=/googleadservices.com/192.168.1.1
    address=/googlesyndication.com/192.168.1.1
    address=/pagead2.googlesyndication.com/192.168.1.1
    address=/video-stats.video.google.com/192.168.1.1
    address=/wintricksbanner.googlepages.com/192.168.1.1
     
  9. TexasFlood

    TexasFlood Network Guru Member

    OK, post if you have any issues once you do get a chance to test it. Atlhough I don't see any obvious problems, I might have fat fingered something else.
     
  10. srouquette

    srouquette Network Guru Member

  11. Jedis

    Jedis LI Guru Member

    Thanks TexasFlood. Appears to be working great. :)
     
  12. TexasFlood

    TexasFlood Network Guru Member

    Your version is very elegant. rhester72's script is a bit hard to read but I recognized it and remembered it was designed to run from administration->scheduler custom script boxes all on one line and controlled by the button on the router face. Easiest thing, although not most elegant, was just to add a filter in-line although taking a lesson from your script to better use pipes would certainly be more concise. I think the second place that 127.0.0.1 is replaced is when a backup file is pulled from cifs. This redundancy could probably be eliminated but I'm lazy and not greatly motivated to fix what seemingly ain't broken, :biggrin:


    Good point, I see Yaqui's script inlcudes instructions for turning it off & on with the router button as well.

    Cool. I recently got a router with a Cisco button, first one for me. I'm not currently using it as my WAN gateway but if I move it there might have to try using the button for this.
     

Share This Page