1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

My ad blocking script for Tomato 1.21

Discussion in 'Tomato Firmware' started by TexasFlood, Aug 12, 2008.

  1. TexasFlood

    TexasFlood Network Guru Member

    Here is my second cut at an ad blocking script (v2) for Tomato v1.21, rolled up tweaks from the last couple of weeks. When I upgraded to Tomato v1.21, my modified version of a script from this forum stopped working. I decided to "roll my own" version, trying to implement the funtionality most important to me in an efficient manner. As stated in my original post, if you think you have seen some part of my script in other scripts, you probably have. I won't deny it, :grin: . I'm posting it in the hope someone else can use all or part of it as I have done with other earlier posts.

    The main script is a bit neater than before. The biggest new additions are command line and router button control scripts added for convenience when needing to turn ad blocking off and on. Most of the script goes on the Init tab with a small component on the WAN Up tab, and the optional button script. I feel that running the script from the init tab gives greater control over when the script is run. You may prefer to run it from the WAN Up tab. Just be aware that the script will run every time WAN Up is run so either add logic to prevent commands from running more than once or confirm it's OK to run the commands more than once.

    *Note - I mention pixelserv below. It is a small perl script that serves up a 1 pixel graphic. Run this on a host. Then set the poisonip variable to the IP address of that host. This will populate that IP into the ad blocking list and result in a much nice looking page when ads are blocked, without all the errors. For more info about pixelserv, see my earlier post. You might also want to check out an interesting little Windows app named AdServer in this dd-wrt forum post that I read about just the other day. I tried AdServer and it works as advertised. It shows blocked sites in a little GUI box which should make it easier to diagnose issues with sites not working properly due to ad blocking. Up to this point I've used software firewall logs to diagnose such problems. Just take the the blocked sites and see which of them you have to remove from the block list to make the web site in question. Like the example below in line 13 of noads.sh, "edgefcs.net" was required to watch usanetwork full TV episodes, so that line removes it from the block list. Just follow that example to selectly remove lines from the block list as you desire.

    The init tab script consists of three basic parts.

    The first part creates the ad blocking control script /root/block for router command line shell use (for those of us without buttons on their routers). It is in /root which is in the default path when logging in via telnet/SSH. Run "block" from telnet or SSH shell.

    Ad blocking control script that is created as /root/block
    This is the ad blocking control script "block". Run with no parameters for help. Ad blocking is on when ad block list is named "dnsmasq.custom", and off when named "dnsmasq.unused". This is controled via the "block" script or the separate router button script. It's pretty simple, not requiring much explaination - a case statment with three cases for the value of the first parameter - on, off or anything else.
    Code:
    adson="/etc/dnsmasq.custom";adsoff="/etc/dnsmasq.unused"
    cycledns="service dnsmasq restart>/dev/null 2>&1"
    if [ ! -f $adson -a ! -f $adsoff ];then echo No ad blocking file, please run /tmp/noads.sh!;exit;fi
    case $1 in
    on) [ -f $adsoff ]&&(mv $adsoff $adson;eval $cycledns);;
    off) [ -f $adson ]&&(mv $adson $adsoff;eval $cycledns);;
    *) echo "Usage: block [on|off]";;
    esac
    echo -n "Ad blocking is ";[ -f $adson ]&&echo on||echo off
    The second part creates script /tmp/noads.sh which creates/updates the ad blocking list from pgl.yoyo.org. Initial state is on but if the blocking state is off when the script is run then that state is retained. Ad sites/domains can be added and removed from the downloaded list. Edit the poisonip variable to change the address to which the ad sites/domains will be redirected.

    Ad file create & update script that is created as /tmp/noads.sh
    Comments are added to the script listing for those interested.
    To conserve memory, unnecessary white space and comments have been removed from the script.
    Code:
    [B]Lines 1-3[/B] - Set the variables, notably the poisonip variable which is the IP address to use for ad block list "redirect IP". For example, you can use 0.0.0.0 or IP address of host running pixelserv or equivalent.
    [B]Lines 4-7[/B] - Get the file from the pgl.yoyo.org website then error checks it. If there is no error, the script continues. If there is an error, it will drop down to line 20 to log the error and exit.
    [B]Lines 8-10[/B] - Add additional web site/domain entries to block. Three examples of ad sites/domains to add are shown. Follow the format of the examples to add more.
    [B]Line 11[/B] - Changes the ad block list IP address to the poisonip variable value. Comment the line out to keep the default IP address of 127.0.0.1.
    [B]Line 13[/B] - Sample of allowing a domain, or more specifically removing it from the ad block list. This entry is needed to watch usanetwork full TV episodes. Follow this format to add more, one entry per line and no blank lines.
    [B]Line 14[/B] - Removes entries like the above example from the ad block list.
    [B]Line 15-16[/B] - Checks to see if the "addoff" file exists. If it does, this means that ad blocking was turned off when the script was run so the ad blocking file is moved over the existing file to preserve both the update and the blocking state that existed when the script was run.
    [B]Line 17[/B] - Restarts dnsmasq if there is an updated "adson" file.
    [B]Line 18[/B] - Logs a sucessful script run and exits.
    01) adsyoyo="/tmp/dnsmasq.yoyo";allow="/tmp/dnsmasq.allow"
    02) adson="/etc/dnsmasq.custom";adsoff="/etc/dnsmasq.unused"
    03) poisonip="0.0.0.0"
    04) yoyourl="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext"
    05) wget -q -O $adsyoyo $yoyourl
    06) if [ $? -eq 0 -a `grep ^address= $adsyoyo|wc -l` -gt 0 ];then
    07) logger Noads pgl.yoyo.org download successful
    08) echo "address=/apn.amazon.com/127.0.0.1">>$adsyoyo
    09) echo "address=/ws.amazon.com/127.0.0.1">>$adsyoyo
    10) echo "address=/astore.amazon.com/127.0.0.1">>$adsyoyo
    11) cat $adsyoyo|sed 's/127.0.0.1/'$poisonip'/g'>$adson;mv -f $adson $adsyoyo
    12) rm -f $allow;rm -f $adson
    13) echo "edgefcs.net">>$allow
    14) [ -f $allow ]&&cat $adsyoyo | grep -v -f $allow>>$adson||mv -f $adsyoyo $adson
    15) rm -f $adsyoyo;rm -f $allow
    16) [ -f $adsoff ]&&mv -f $adson $adsoff
    17) [ -f $adson ]&&service dnsmasq restart>/dev/null 2>&1
    18) logger Noads script ran successfully
    19) else
    20) logger Noads pgl.yoyo.org download failed, script exited
    21) fi
    The third part waits for the WAN Up tab script to create /tmp/wanisup before proceeding then does an intial run of the noads script, schedules recurring runs of the sciript and optionally turns ad blocking off (uncomment the last line if you want the initial state to be off). The cru statement sets up a cron schedule to run the script every Sunday at midnight, change or remove this line to suit your preferences.

    Paste into Init tab
    Includes escape "\" characters to protect "$" and "`" characters from being dropped, the "\" characters will not be in the final script.
    There is one comment line with my pseudonym, the script version and date. This line can, of course, be removed as well.
    Code:
    # TexasFlood Ad blocking scripts for Tomato v1.21, v2 8-25-2008
    cat <<END>/root/block
    adson="/etc/dnsmasq.custom";adsoff="/etc/dnsmasq.unused"
    cycledns="service dnsmasq restart>/dev/null 2>&1"
    if [ ! -f \$adson -a ! -f \$adsoff ];then echo No ad blocking file, please run /tmp/noads.sh!;exit;fi
    case \$1 in
    on) [ -f \$adsoff ]&&(mv \$adsoff \$adson;eval \$cycledns);;
    off) [ -f \$adson ]&&(mv \$adson \$adsoff;eval \$cycledns);;
    *) echo "Usage: block [on|off]";;
    esac
    echo -n "Ad blocking is ";[ -f \$adson ]&&echo on||echo off
    END
    chmod +x /root/block
    cat <<END>/tmp/noads.sh
    adsyoyo="/tmp/dnsmasq.yoyo";allow="/tmp/dnsmasq.allow"
    adson="/etc/dnsmasq.custom";adsoff="/etc/dnsmasq.unused"
    poisonip="0.0.0.0"
    yoyourl="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext"
    wget -q -O \$adsyoyo \$yoyourl
    if [ \$? -eq 0 -a \`grep ^address= \$adsyoyo|wc -l\` -gt 0 ];then
    logger Noads pgl.yoyo.org download successful
    echo "address=/apn.amazon.com/127.0.0.1">>\$adsyoyo
    echo "address=/ws.amazon.com/127.0.0.1">>\$adsyoyo
    echo "address=/astore.amazon.com/127.0.0.1">>\$adsyoyo
    cat \$adsyoyo|sed 's/127.0.0.1/'\$poisonip'/g'>\$adson;mv -f \$adson \$adsyoyo
    rm -f \$allow;rm -f \$adson
    echo "edgefcs.net">>\$allow
    [ -f \$allow ]&&cat \$adsyoyo | grep -v -f \$allow>>\$adson||mv -f \$adsyoyo \$adson
    rm -f \$adsyoyo;rm -f \$allow
    [ -f \$adsoff ]&&mv -f \$adson \$adsoff
    [ -f \$adson ]&&service dnsmasq restart>/dev/null 2>&1
    logger Noads script ran successfully
    else
    logger Noads pgl.yoyo.org download failed, script exited
    fi
    END
    chmod +x /tmp/noads.sh
    while [ ! -f /tmp/wanisup ];do ( sleep 1);done
    rm /tmp/wanisup
    logger WAN is up, starting noads script initial run
    /tmp/noads.sh>/dev/null 2>&1
    cru a BlockWebAds "2 0 * * 0 /tmp/noads.sh"
    #/root/block off>/dev/null 2>&1
    Paste into WAN Up tab
    Code:
    #Create file wanisup to flag that WAN is up then remove these lines from WAN Up script so aren't run again
    touch /tmp/wanisup
    sed -i '/wanisup/d' /tmp/script_wanup.sh
    "Run Custom Script" button code for those who can use it
    Structure borrowed from rhester72s voluminous post. I didn't test it as my router doesn't have a button, but it should work as rhester described.
    Quick (<3 seconds) button hit to enable/disable ad blocking (optional):
    Set 0-2 seconds to "Run Custom Script", and paste the following into the Custom Script box:
    Code:
    [ $1 -lt 3 ]&&([ -f /etc/dnsmasq.custom ]&&mv /etc/dnsmasq.custom /etc/dnsmasq.unused||mv /etc/dnsmasq.unused /etc/dnsmasq.custom;service dnsmasq restart)
     
  2. fyellin

    fyellin LI Guru Member

    Is there a reason you "sleep" in the init tab waiting for the WAN to be up, rather than just putting that piece of code code in the wan-up section.

    You can still use
    if [ - f "/tmp/wanisup"]
    to ensure that it the code is executed only once.
     
  3. rhester72

    rhester72 Network Guru Member

    Just my experience, but I can't get logger to talk to syslog at all from the init script (at least on boot) - I figured syslog loads after init. Your experience is different, I suppose, based on the scripts?

    Rodney
     
  4. TexasFlood

    TexasFlood Network Guru Member

    Could do it either way.

    What I had in mind was that I had more control on the Init tab. I know that /tmp/noads.sh is written only once and the commands execute only once, rather than every time the WAN Up script is run.

    OK, the WAN Up script realistically won't run that often, I admit.

    Maybe it's silly but it's what I got in my head to do and I got it to work, :wink:

    It would work just fine on the WAN Up tab. You could sure it runs only once by checking the test file as you described. Technically it might not even matter if it did run more than once. The script woiuld over-write itself, the cron job would get added again, the script would repeat it's intial run and log it. Really no harm done. I'd probably want to suppress at least the initial script run and log entry. But hey, I was being such a control freak that I insisted it all run only one! So don't go by me, :grin:

    6 of one, 1/2 dozen of the other so do it whichever way you prefer.
     
  5. TexasFlood

    TexasFlood Network Guru Member

    Thing is, nothing actually logs until after the WAN comes up anyway, the first logger statement is after the test file is written on the WAN Up tab. OK, I admit, in retrospect it might be a little silly to do it that way, but I had my mind made up and coudn't talk myself out of it! :cool: Maybe I'll rewrite it differently tomorrow :biggrin:
     
  6. rhester72

    rhester72 Network Guru Member

    Believe it or not, init.sh can run more than once, but I don't recall the circumstances. I saw it myself.

    Rodney
     
  7. TexasFlood

    TexasFlood Network Guru Member

    Bump to note script update to v2 in the first post of this thread.

    Since it does what I was initially looking for, I'll quit tweaking it, at least for a while :grin: . Until I think of something else it should do, :wink:
     

Share This Page