Basically, it blocks all ads and trash websites via the DNS cache poisoning method Updated: 08 July 2008 v2.1 This script differs from others: Automatic Update (every 6 hours) (configures itself) Multipe Hosts Files (Combines and Removes Duplicates) Removes Duplicate Hosts Easy to use, just paste the code. Only runs once WAN is started. Low on memory (no files are kept once the script runs). Works on all versions of tomato 1.5 and newer. Frequently Updated Code. NO extra files need to be created Does Not require jffs Optimises dnsmasq Single Script Uses (either): MVPS HOSTS file: http://www.mvps.org/winhelp2002/hosts.txt (~18,500 records, 680 Kbyte) PGL YOYO: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts (~2,200 records, 68 Kbyte) hosts-file.net: http://www.it-mate.co.uk/downloads/hosts.txt (~53,000 records, 1.5 Mbyte) The Hosts File Project: http://hostsfile.mine.nu/Hosts (~102,000 records, 3.0 Mbyte) ***requires 6mb free memory*** Version History: v1.1 added: whitelisting, and defaulted the hosts file. v1.2 added: varibles. v1.3 removed: caching of source hosts file to lower memory usage v1.4 fix: correct generation of files, set ips to 0.0.0.0, neatly stop dnsmasq v1.5 removed: generated file after its loaded (frees up memory) v1.6 added: Allow for 2 methods of blocking: dnsmasq (default) and hostfile v1.7 fix: Code Optimisations v1.8 Removed: Removal of Duplicates (dnsmasq already takes care of it, was causing heavy processing load) v1.9 fix: Memory Optimisations, Removal of Duplicates added (new resource friendly method) v1.9.5 fix: Default dnsmasq config values added to generated config v2.0 added: Automatic Update (every 6 hours), Optimising of dnsmasq, error checking, generic source file cleanup, multiple host files, Log Messages v2.1 fix: Error with updated hosts file Note Max Script Size: 4000 Bytes mvps.org and pgl.yoyo hosts combined (can combine others.. if ur router has the memory). Installation Instructions: Go to your Tomato Web Interface ( default: http://192.168.1.1 ) Navigate: Administration -> Scripts -> WAN Up Copy the contents of the box below and paste it into the empty area (you do not have to chnage anything) Click the SAVE button at the bottom of the box. To enable the script: restart the router or disconnect and reconnect the WAN/internet connection. To test: navigate to a website with lots of adverts. (eg: http://pagead2.googlesyndication.com ) \/ COPY THE FOLLOWING \/ (inside the box) Code: #!/bin/sh ## Adblock script [Version 2.1 | 08 July 2008 | 3778 bytes] ## ## Created by Adrian Jon Kriel: root-AT-extremecooling-DOT-org ## ## tomato WAN Up script ## ## 0 = disable ## 1 = enable ## (1) = default value ## optimising of dnsmasq, (1) eval OPTDNSMASQ="1" ## automatic updating, (1) eval AUTOUPDATE="1" ## MVPS HOSTS ~18,500 lines, 680 Kbyte, (1) eval MVPSSOURCE="1" ## pgl.yoyo.org ~2,200 lines, 68 Kbyte, (1) eval PGLSOURCE="1" ## hosts-file.net ~53,000 lines, 1.5 Mbyte, (0) eval HSFSOURCE="0" ## Hosts File Project ~102,000 lines, 3.0 Mbyte ***6+mb free memory***, (0) eval HFPSOURCE="0" ## ## varibles ## location of temp file, (/tmp/hosts) eval GENFILE="/tmp/hosts" ## redirect ip, (0.0.0.0) eval REDIRECTIP="0.0.0.0" ## sources eval MVPSOURCEFILE="http://www.mvps.org/winhelp2002/hosts.txt" eval PGLSOURCEFILE="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" eval HSFSOURCEFILE="http://www.it-mate.co.uk/downloads/hosts.txt" eval HFPSOURCEFILE="http://hostsfile.mine.nu/Hosts" if ping -c 1 yahoo.com ; then eval GOTSOURCE="0" echo "" > $GENFILE ## download if [ "$MVPSSOURCE" = "1" ] ; then if wget $MVPSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK Downloaded $MVPSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Failed $MVPSOURCEFILE fi fi if [ "$PGLSOURCE" = "1" ] ; then if wget $PGLSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK Load $PGLSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Fail $PGLSOURCEFILE fi fi if [ "$HSFSOURCE" = "1" ] ; then if wget $HSFSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK load $HSFSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Fail $HSFSOURCEFILE fi fi if [ "$HFPSOURCE" = "1" ] ; then if wget $HFPSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK Load $HFPSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Fail $HFPSOURCEFILE fi fi if [ "$GOTSOURCE" = "1" ]; then logger ADBLOCK Got Source Files #FREE MEMORY! service dnsmasq stop killall -9 dnsmasq logger ADBLOCK Ignor Fail Safe ##strip source file sed -i -e '/^[0-9A-Za-z]/!d' $GENFILE sed -i -e '/%/d' $GENFILE sed -i -e 's/[[:cntrl:][:blank:]]//g' $GENFILE sed -i -e 's/^[ \t]*//;s/[ \t]*$//' $GENFILE ## dnsmasq, sanitize, optimised sed -i -e 's/[[:space:]]*\[.*$//' $GENFILE sed -i -e 's/[[:space:]]*\].*$//' $GENFILE sed -i -e '/[[:space:]]*#.*$/ s/[[:space:]]*#.*$//' $GENFILE sed -i -e '/^$/d' $GENFILE sed -i -e '/127.0.0.1/ s/127.0.0.1//' $GENFILE sed -i -e '/^www[0-9]./ s/^www[0-9].//' $GENFILE sed -i -e '/^www./ s/^www.//' $GENFILE ## remove duplicates (resource friendly) cat $GENFILE | sort -u > $GENFILE.new mv $GENFILE.new $GENFILE ## format sed -i -e 's|$|/'$REDIRECTIP'|' $GENFILE sed -i -e 's|^|address=/|' $GENFILE ## load values from dnsmasq config cat /etc/dnsmasq.conf >> $GENFILE ## optimise dnsmasq if [ "$OPTDNSMASQ" = "1" ] ; then cat >> $GENFILE <<EOF cache-size=2048 log-async=5 EOF fi ## remove/whitelist websites ## removes 3 websites (aa.com, bb.com, cc.com) ## remove the # and edit the website urls. sed -i -e '/aa.com/d' $GENFILE sed -i -e '/bb.com/d' $GENFILE sed -i -e '/cc.com/d' $GENFILE ## apply blacklist dnsmasq --conf-file=$GENFILE ## failsafe added dnsmasq logger ADBLOCK Ignor Fail Safe ## dev info logger ADBLOCK Unique Hosts Blocked $(awk 'END { print NR }' $GENFILE) else logger ADBLOCK Error Not Downloaded fi else logger ADBLOCK Error No Internet fi ## remove the generated files rm $GENFILE* ## automatic update if [ "$AUTOUPDATE" = "1" ] ; then ## script exists if [ -x /tmp/script_wanup.sh ] ; then cru a UpdateAdlist "0 6,12,18,0 * * * /tmp/script_wanup.sh >/dev/null 2>&1" fi fi ## the end If you want to disable the blacklisting login to your router via ssh and execute this command. This will disable the blacklisting that the script generates, thus whitelisting everything. It will not restart the router, as no restart is required. Code: dnsmasq restart to re-enable the hosts file. (blacklist domains) login to your router via ssh and execute this command. Code: /tmp/script_wanup.sh
This has been discussed before. Here's another script, which automatically updates the the ad block list on a routine basis. http://www.linksysinfo.org/forums/showthread.php?t=53133 Search, and you will find others as well...
Ad Blocking Isnt this what the access restriction entries are for? I have a number of keywords I block in HTTP Request, these keywords are based on my browsing habits as opposed to a 'catch all, but never 100% list' approach. One thing I have noticed though, and could some one please confirm, if I select HTTP Requested Files ( block swf/cab etc), this appear to block ALL swf/cab's, not just the keyworded sites. Here are the top three stats!!!!, no surprise here. DOUBLECLICK-NET US 15038 GOOGLE US 8732 MCI Communications US 2476
You absolutely can use access restrictions for that, however, the MVPS list i've been using on my router for over a year is updated regularly, and currently contains over 17,000 sites which are blocked. (Though access restrictions are implemented differently.) Read through the many existing threads on the subject to understand the pros and cons of each method. This has all been discussed before. - Mike
Wow, I didn't know this was possibly. Never even thought of it, so this post is helpful. I use a hosts file on all my PCs, this might be an improvement. One question though, once in a while, I want to get to a site that's listed in the hosts file. I have a little batch file on the PCs now I run. How would I temporarily disable the hosts file if I go with this method? Also, I use the one at hosts-file.net, but the MVP one, but I guess that's possible too, right?
updated: added whitelisting support. #// remove/whitelist websites #// this will remove 3 websites (abcd.com, efgh.com, ijkl.com) from the hosts file (whitelists them). #// remove the # and edit the website urls. #sed -e '/abcd.com/d' -e '/efgh.com/d' -e '/ijkl.com/d' /tmp/hosts
added varibles to allow for easier customisation: added option to use host-file.net (~57 000 blocked addresses)
I really like the Whitelist option I stopped using the other scripts because my wife uses one of the sites on there and I didn't know how to get around it.
This is excellent. Still, sometimes I only want to quickly, temporarily get to a site that's listed in the hosts file. What would be the way to temporarily deactivate the hosts file? Would I have to reboot the router to make that happen or is there an easier way?
You could SSH into the router, use "mv /tmp/hosts /tmp/hosts-disabled" (without quotes) to rename the hosts file, and then use "service dnsmasq restart" to restart the program that uses the host file. Because you renamed the hosts file, it will be ignored. When you are done, do the opposite of step 1 which is "mv /tmp/hosts-disabled /tmp/hosts" and repeat step 2 with "service dnsmasq restart" It's probably not easier than just restarting your router though.
Hi the easiest and more correct way. Is to login to ssh and execute this command. This will disable the hosts file that the script generates, thus whitelisting everything. It will not restart the router, as no restart is required. Code: killall -9 dnsmasq dnsmasq to re-enable the hosts file. (blacklist domains) Code: #// varibles eval GENHOSTSFILE="/tmp/hosts" killall -9 dnsmasq dnsmasq --addn-hosts=$GENHOSTSFILE updated to v1.3
here are the results: black listing (script running per usual) # ping ad.a8.net PING ad.a8.net (0.0.0.0): 56 data bytes 64 bytes from 127.0.0.1: seq=1 ttl=64 time=0.854 ms whitelisting (script disabled) # killall -9 dnsmasq # dnsmasq # ping ad.a8.net PING ad.a8.net (202.177.198.92): 56 data bytes 64 bytes from 202.177.198.92: seq=0 ttl=41 time=843.319 ms blacklistsing (script re-enabled) # #// varibles # eval GENHOSTSFILE="/tmp/hosts" # killall -9 dnsmasq # dnsmasq --addn-hosts=$GENHOSTSFILE # ping ad.a8.net PING ad.a8.net (0.0.0.0): 56 data bytes 64 bytes from 127.0.0.1: seq=0 ttl=64 time=1.003 ms
Well, in the case, since it's something I do fairly frequently, if it's hard to disable temporarily or I need to restart the router, maybe I'll just continue to use it on the locally 3 PCs I have and forget about doing it in the router.
Major Update: Added: The Hosts File Project: http://hostsfile.mine.nu/Hosts Allow for 2 methods of blocking: dnsmasq (default) and hostfile Converts a block hosts file to dnsmasq config file ( conversion code inspired by: YAQUI: http://www.linksysinfo.org/forums/showpost.php?p=307473&postcount=31 )
I prefer the DNS cache poisoning method because it works across multiple web browsers and eliminates most/all web advertisements under them. Not to mention I have several different machines with different operating systems. Another nice thing with DNS Cache Poisoning is that it works across all my applications, Not just webbrowsers. No matter what browser I'm currently using at the moment (dillo, opera, ie, ff, etc) ads are blocked.
Yeah, what he said... I have 6 machines with different OSes. They are all configured via the router with DHCP, and all DNS requests go to the router to filter. I don't have to worry about making sure all PC's are up to date with the latest list. I use the MVPS list on the router, and I'm good to go, across the board.
Certainly having cross platform management is good way to go. I've just started poking around and this may already be somewhere but is there a step by step for dumbies instruction somewhere? Also wondering if using a jiffs storage section in the router would be something that one could use for these host files?
Hi Just follow the steps Ive designed mine so it does not need jffs or any extra files. My goal is for it to work out the box. (options in the file are for us geeks)
Updated to 1.9 Duplicates Removed (resource friendly method) dnsmasq stopped to allow for more memory for processing
So I'm copying the script in the first or second posts, paste it into the WAN up as is? Save and reboot? Probably could at least disable adblock in Firefox to make sure I'm "testing" the script? Jim Edit: "options in the file are for us geeks" I completely agree and don't mind being a non-geek. However IF us non-geeks get step by step do this, do that, kind of direction, it is amazing just how geeky we can actually be....
Hit a few sites to test it. Sites I know cram ads down your throat... No I mean sites who costs are offset by advertisement.... Anyway and under the heading of "Who and the H*** does this rookie think he is..." Having now seen the impact and the empty boxes of didn't connect, etc., displays, is there a way to have some kind of default graphic image display in those blocked ad spaces? See what I mean, "Who and H*** does this guy think he is...."
This script seems to break local DNS resolution. For instance, i use static DHCP to assign local IPs and names in tomato. This allows me to simply type "phone" into a browsers address bar and tomato resolves it as the local IP that my SIP device is attached to. After running this script this no longer works. Previously i was running a different script using the dnsmasq formatted hosts list from pgl.yoyo.org. This has far fewer entries than the lists in your script but i was also able to use local DNS resolution with it. As far as i can tell both scripts are presenting the host lists in exactly the same format so am at a loss as to why local DNS no longer works with your script. If i do a "service dnsmasq restart" local DNS resolution is restored but obviously at the cost of ad blocking. Could it be that there are so many entries in the hosts file that dnsmasq can not take them all and drops some? If local entries are added after that it might explain why they no longer work.
been having issues with the lack of memory, and testing. It blocks ~15,500 domains. RE: MiddlingMan . The issue is that i generate a complete stand alone config. Ill upload a fix in a short while. (will be version 1.95) (basically itll copy the contents of the default config file into the one it generates) that *should* fix the errors. RE: jsmiddleton4 . Already working on something... currently i have a webserver on my lan, hence the option to set the redirection ip. (ill post more on this later.. still looking for something thats easy to implement) RE: jsmiddleton4 . Soz for the instructions.. ill improve them.
May have nothing to do with this add blocking script but did want to at least bring it up. I'm betting a dhcp/dnsmaq error message in my log when running the ad blocking script. Using the 1.95 one by the way. I'm getting a failed to start message with address already assigned/bound message. Speculating about dnsmasq thing already running then trying to run again. Its here: http://www.linksysinfo.org/forums/showthread.php?p=327197#post327197 Again NOT saying either has anything to do with either. Just that it "could be" so wanted to point it out. Jim
VERSION 2 !!! Automatic Update (every 6 hours) Optimising of dnsmasq Error checking Generic source file cleanup MULTIPLE HOSTS files !! (combines and removes the duplicates.) Log Messages Log Errors: Tells u what to ignore
xc.... Cool. And I had/have no idea if it was good or bad, bug or not. Just passing on my observation. And just checking, did you just add that ADBLOCK ignore line? I don't remember seeing it before. "Tells u what to ignore" Now THAT'S handy. Can I suggest some entries to the "tell you to ignore" list? Thanks for all your hard work by the way. Edit: Just noticed the answer to my own question in the other thread. Is there a way, and again having a mindset that even thinks about "fail safe" in the script is quite amazing and thanks for putting that kind of safety in your script.
xc.... Not sure if this is even a problem but its another one of those things I wanted to bring forward. And since as a feature sometimes it doesn't work well anyway its hard to tell exactly what is going on. Running your 2.0 script. I can not seem to d/l any new favicons. Of course the problem with that as a symptom is you often can't anyway even though a site has a favicon and it shows up in address bar. And certainly not a big deal. Just wondering mostly is there any unintended blocking of favicons with the ad blocking?
nope.. it can only block domains.. Im running it, and all my fav icons are working fine. Including the one for this website.
Thanks XC... For me with Firefox I'm finding the favicon thing is hit or miss so hard for me to tell. I'll see the favicon in the address bar but it doesn't load to bookmark. Really seems silly but hey, I like my favicons....
I think one of the hosts files has an error as I receive this "Jul 7 12:56:35 unknown daemon.crit dnsmasq[830]: error at line 10860 of /tmp/hosts" It uses these files: "Jul 7 12:55:59 unknown user.notice root: ADBLOCK Downloaded http://www.mvps.org/winhelp2002/hosts.txt Jul 7 12:56:00 unknown user.notice root: ADBLOCK Load http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" Right now, the adblocking stopped working.
Please restart ur router and try again.. Has the script worked for u.. or its this the first time trying ? Ive got no error in those files
There is currently an error in the MVPS file. They crop up occasionally. Not sure what's breaking this script, though. The script I'm using (from another thread) just reports the error and uses the rest of the file, no problem. The adblocking still works for me. Code: Jul 7 13:55:28 Router user.notice root: DOWNLOADED http://www.mvps.org/winhelp2002/hosts.txt Jul 7 13:55:28 Router daemon.err dnsmasq[300]: bad name at /etc/hosts line 6218 Jul 7 13:55:28 Router daemon.err dnsmasq[300]: bad name at /etc/hosts line 6218 Jul 7 13:55:28 Router daemon.err dnsmasq[300]: bad name at /etc/hosts line 6219 Jul 7 13:55:28 Router daemon.err dnsmasq[300]: bad name at /etc/hosts line 6219 Jul 7 13:55:29 Router daemon.info dnsmasq[300]: read /etc/hosts - 17948 addresses Jul 7 13:55:29 Router daemon.info dnsmasq[300]: read /etc/hosts.dnsmasq - 11 addresses
It worked perfect for 2 weeks. It's only today that it stopped working. I restarted the router and it still no go. And like Mraneri, I noticed that the problem is with http://www.mvps.org/winhelp2002/hosts.txt, I disabled all the other and only used this one. And btw, the files: http://www.it-mate.co.uk/downloads/hosts.txt http://hostsfile.mine.nu/Hosts are ridiculous! they block a lot of the well known website including google.com and nytimes.com
I would really like to get to the bottom of this.. Because on my router im not getting the error.. Can u please paste ur code here.. and im going to get this sorted asap.
Hey xcooling, Here's the code: Code: #!/bin/sh ## Adblock script [Version 2.0 | 16 June 2008 | 3997 bytes] ## ## Created by Adrian Jon Kriel: root-AT-extremecooling-DOT-org ## ## tomato WAN Up script ## ## 0 = disable ## 1 = enable ## (1) = default value ## enable blocking via hostfile, (0) eval HOSTFILE="0" ## optimising of dnsmasq, (1) eval OPTDNSMASQ="1" ## automatic updating, (1) eval AUTOUPDATE="1" ## MVPS HOSTS ~18,500 lines, 680 Kbyte, (1) eval MVPSSOURCE="1" ## pgl.yoyo.org ~2,200 lines, 68 Kbyte, (1) eval PGLSOURCE="1" ## hosts-file.net ~53,000 lines, 1.5 Mbyte, (0) eval HSFSOURCE="0" ## Hosts File Project ~102,000 lines, 3.0 Mbyte ***6+mb free memory***, (0) eval HFPSOURCE="0" ## ## varibles ## location of temp file, (/tmp/hosts) eval GENFILE="/tmp/hosts" ## redirect ip, (0.0.0.0) eval REDIRECTIP="0.0.0.0" ## sources eval MVPSOURCEFILE="http://www.mvps.org/winhelp2002/hosts.txt" eval PGLSOURCEFILE="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" eval HSFSOURCEFILE="http://www.it-mate.co.uk/downloads/hosts.txt" eval HFPSOURCEFILE="http://hostsfile.mine.nu/Hosts" if ping -c 1 yahoo.com ; then eval GOTSOURCE="0" echo "" > $GENFILE ## download if [ "$MVPSSOURCE" = "1" ] ; then if wget $MVPSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK Downloaded $MVPSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Failed $MVPSOURCEFILE fi fi if [ "$PGLSOURCE" = "1" ] ; then if wget $PGLSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK Load $PGLSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Fail $PGLSOURCEFILE fi fi if [ "$HSFSOURCE" = "1" ] ; then if wget $HSFSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK load $HSFSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Fail $HSFSOURCEFILE fi fi if [ "$HFPSOURCE" = "1" ] ; then if wget $HFPSOURCEFILE -O - >> $GENFILE ; then logger ADBLOCK Load $HFPSOURCEFILE eval GOTSOURCE="1" else logger ADBLOCK Fail $HFPSOURCEFILE fi fi if [ "$GOTSOURCE" = "1" ]; then logger ADBLOCK Got Source Files #FREE MEMORY! service dnsmasq stop killall -9 dnsmasq logger ADBLOCK Ignore Fail Safe ##strip source file sed -i -e '/^[0-9A-Za-z]/!d' $GENFILE sed -i -e '/%/d' $GENFILE sed -i -e 's/[[:cntrl:][:blank:]]//g' $GENFILE if [ "$HOSTFILE" = "1" ] ; then ## hostfile ## santize and format sed -i -e '2,$s/127.0.0.1/'$REDIRECTIP'/g' $GENFILE sed -i -e '/%/d' $GENFILE sed -i -e '/[[:space:]]*#.*$/ s/[[:space:]]*#.*$//' $GENFILE else ## dnsmasq ## sanitize, optimised sed -i -e '/[[:space:]]*#.*$/ s/[[:space:]]*#.*$//' $GENFILE sed -i -e '/^$/d' $GENFILE sed -i -e '/127.0.0.1/ s/127.0.0.1//' $GENFILE sed -i -e '/^www[0-9]./ s/^www[0-9].//' $GENFILE sed -i -e '/^www./ s/^www.//' $GENFILE ## remove duplicates (resource friendly) cat $GENFILE | sort -u > $GENFILE.new mv $GENFILE.new $GENFILE ## format sed -i -e 's|$|/'$REDIRECTIP'|' $GENFILE sed -i -e 's|^|address=/|' $GENFILE ## load values from dnsmasq config cat /etc/dnsmasq.conf >> $GENFILE ## optimise dnsmasq if [ "$OPTDNSMASQ" = "1" ] ; then cat >> $GENFILE <<EOF cache-size=2048 log-async=5 EOF fi fi ## remove/whitelist websites ## removes 3 websites (aa.com, bb.com, cc.com) ## remove the # and edit the website urls. sed -i -e '/skins.be/d' $GENFILE sed -i -e '/theglobeandmail.com/d' $GENFILE #sed -i -e '/aa.com/d' $GENFILE #sed -i -e '/bb.com/d' $GENFILE #sed -i -e '/cc.com/d' $GENFILE ## apply blacklist if [ "$HOSTFILE" = "1" ] ; then ## hostfile dnsmasq --addn-hosts=$GENFILE else ## dnsmasq dnsmasq --conf-file=$GENFILE fi ## failsafe added dnsmasq logger ADBLOCK Ignore Fail Safe ## dev info logger ADBLOCK Unique Hosts Blocked $(awk 'END { print NR }' $GENFILE) else logger ADBLOCK Error Not Downloaded fi else logger ADBLOCK Error No Internet fi ## remove the generated files rm $GENFILE* ## automatic update if [ "$AUTOUPDATE" = "1" ] ; then ## script exists if [ -x /tmp/script_wanup.sh ] ; then cru a UpdateAdlist "0 6,12,18,0 * * * /tmp/script_wanup.sh >/dev/null 2>&1" fi fi ## the end And here's the tomato log, I'm using the forked MLPPP tomato firmware from dslreports http://www.dslreports.com/forum/r20714586-TomatoMLPPP-v2-beta-2-released Code: ul 7 15:31:07 unknown local2.notice pppd[2494]: Link terminated. Jul 7 15:31:11 unknown user.notice root: ADBLOCK Downloaded http://www.mvps.org/winhelp2002/hosts.txt Jul 7 15:31:13 unknown user.notice root: ADBLOCK Load http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts Jul 7 15:31:13 unknown user.notice root: ADBLOCK Got Source Files Jul 7 15:31:13 unknown daemon.info dnsmasq[2508]: exiting on receipt of SIGTERM Jul 7 15:31:13 unknown user.notice root: ADBLOCK Ignore Fail Safe Jul 7 15:31:47 unknown daemon.crit dnsmasq[2666]: error at line 10860 of /tmp/hosts Jul 7 15:31:47 unknown daemon.crit dnsmasq[2666]: FAILED to start up Jul 7 15:31:47 unknown user.notice root: ADBLOCK Ignore Fail Safe Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: started, version 2.41 cachesize 150 Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: compile time options: no-IPv6 GNU-getopt no-RTC no-ISC-leasefile no-DBus no-I18N no-TFTP Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: DHCP, IP range 192.168.10.100 -- 192.168.10.149, lease time 1d Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: reading /etc/resolv.dnsmasq Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: using nameserver 206.53.60.10#53 Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: using nameserver 206.53.48.23#53 Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: read /etc/hosts - 0 addresses Jul 7 15:31:47 unknown daemon.info dnsmasq[2670]: read /etc/hosts.dnsmasq - 3 addresses Jul 7 15:31:50 unknown user.notice root: ADBLOCK Unique Hosts Blocked 17453
I don't know if this is appropriate or not since I didn't see any other thank you posts. But Thank you, xcooling. This is amazing, especially with your vigilant updates
I use the one here (ok. I'm biased.. I wrote it...): http://www.linksysinfo.org/forums/showthread.php?t=53133 Note, the script downloaded the file (as it does every week) from MVPS on Friday last week (7/4), and the error wasn't present. However, when I ran it manually this afternoon, the error was there, so the error popped up on the MVPS site sometime between Friday and today. Either way, this script doesn't seem affected by the errors. I've seen other DL's with errors in the past, and they didn't break the script either.
Hi sorry for the delay, wanted to make sure it was 100% Ive uploaded the fixed version 2.1. now compatible with the new format of the host files. Also did a few code optimisations. Please let me know. Thanks
Thank you for the update, the script works perfectly now! I wish all the programmers were quick to respond like you.
You may want to see my ALL-U-NEED script for the proper sed formatting. (which is somewhat similar to what you are doing and the K.I.S.S. theory only mine is slightly longer and uses HERE document.) I went through and found what was causing the error, it had to do with the carriage returns and 2 bracketed comments which they (forgot?) to space out properly. Maybe you can adapt my sed statements to your script.
um... it was [ ] these that caused the error re the sed formatting, the reason they are written differently is they process faster. I found an interesting whitepaper on sed optimisations an did some testing.
Right, if you read my post (cough cough wow yes flu season still bad) I mentioned the bracketed comments along with removing carriage returns. Plus I wrote my own sed statements, I wasn't referring to yours : I was replying to mraneri, just trying to help him out. Since we seem to like the simplified scripts which I had originally based mine off his.
What would be nice is instead of your browser showing lots of page loads errors where the ads should be is just a nice simple white page. Now the only way i can think about doing this is to run a webserver on a pc somewhere, with a blank page and direct them all to that. Any other ideas?
Would it not be possible for the router itself to host as supply a single pixel image or plain white (black etc) page to be redirected to?
Might slow it down, but would certainly improve it aesthetically. Took a look at what images are already in the firmware. There is a pretty small one called bar1.gif, just not quite sure how to repoint to it.
I am somewhat partial to http://192.168.1.1/spin.gif , but I definitely lack the skills to pull it off.
I've been reading up about using a .htaccess file to point or redirect a 404 error. I think this might be workable somehow. Thoughts?
Remember; the web server on the Linksys is NOT a standard server; it's a fairly paired down server and I dont believe it contains support for a .htaccess file. I don't have the source to Tomato on this machine right now, so can't go browsing through it....
In anycase, as a future enhancement, the ability to redirect the 404 errors at the router would be sweet.
Error in tmp/hosts line 18020....tried to restart...wipe out the file...same thing. Any idea what's going on? Thx
I got an error recently for specifying cache-size and log-async options in the Dnsmasq Custom Configuration since the script was adding these options as well. Might not be the same error but it's the only one I've seen.
So I managed to pull this off using one of my linux file/www servers. Basically, I modified the add-blocking script to use the internal address for the linux web server. I created a file called 404err.html and then modified the apache2.conf file to use that specific page as the the 404 error page. Now I can edit the 404err.html file with whatever I would like to replace the banner ads. EDIT: It would still be nice to be able to do this on the router as a self contained system.
TexasFlood: Do you get the error if you disable my o ptimisations of dnsmasq ( ie. in my script change this: eval OPTDNSMASQ="1" to eval OPTDNSMASQ="0" ) s0dhi: could u please post your 404err.html and your apache2.conf ? Im working on a update, will have more posted soon as im done with it.
Thanks, I'll try it when I get home and see. It's working OK for now, I just removed those conflicting options from my dnsmasq custom configuration. FYI. Since I upgraded my main router to Victek's latest builds using DNSmasq 2.45, it seems like I've seen a couple of occasions when the script seemed to run fine but wasn't blocking sites. First it happened with "Tomato RAF 1.20.8502ND" and I couldn't seem to resolve it. So I switched to "Tomato RAF 1.20.8002" and everything seemed to be working. Late last night seemed like again sites weren't getting blocked but a reboot fixed it. Unfortunately I haven't had the time to pay proper attention to make sure I wasn't imagining it. Just an FYI that the latest DNSmasq -may- have introduced a problem with the script. Might just be my sleep deprived imagination, I'll let you know if I see it again.
BTW, I am using the "all you need" blocking script from yaqui. I modified it such that the command that replaces 127.0.0.1 with 0.0.0.0 now uses 192.168.1.5. Below is a snip of the apache2.conf that has the relevant line uncommented (the 404 error one). I can post the entire file, but I think it might be overkill. Code: # # Customizable error responses come in three flavors: # 1) plain text 2) local redirects 3) external redirects # # Some examples: #ErrorDocument 500 "The server made a boo boo." #ErrorDocument 404 /missing.html #ErrorDocument 404 "/cgi-bin/missing_handler.pl" #ErrorDocument 402 http://www.example.com/subscription_info.html # # # Putting this all together, we can internationalize error responses. # # We use Alias to redirect any /error/HTTP_<error>.html.var response to # our collection of by-error message multi-language collections. We use # includes to substitute the appropriate text. # # You can modify the messages' appearance without changing any of the # default HTTP_<error>.html.var files by adding the line: # # Alias /error/include/ "/your/include/path/" # # which allows you to create your own set of files by starting with the # /usr/share/apache2/error/include/ files and copying them to /your/include/path/, # even on a per-VirtualHost basis. The default include files will display # your Apache version number and your ServerAdmin email address regardless # of the setting of ServerSignature. # # The internationalized error documents require mod_alias, mod_include # and mod_negotiation. To activate them, uncomment the following 30 lines. # Alias /error/ "/usr/share/apache2/error/" # # <Directory "/usr/share/apache2/error"> # AllowOverride None # Options IncludesNoExec # AddOutputFilter Includes html # AddHandler type-map var # Order allow,deny # Allow from all # LanguagePriority en cs de es fr it nl sv pt-br ro # ForceLanguagePriority Prefer Fallback # </Directory> # # ErrorDocument 400 /error/HTTP_BAD_REQUEST.html.var # ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var # ErrorDocument 403 /error/HTTP_FORBIDDEN.html.var # ErrorDocument 404 /error/HTTP_NOT_FOUND.html.var ErrorDocument 404 /404err.html # ErrorDocument 405 /error/HTTP_METHOD_NOT_ALLOWED.html.var # ErrorDocument 408 /error/HTTP_REQUEST_TIME_OUT.html.var # ErrorDocument 410 /error/HTTP_GONE.html.var # ErrorDocument 411 /error/HTTP_LENGTH_REQUIRED.html.var # ErrorDocument 412 /error/HTTP_PRECONDITION_FAILED.html.var # ErrorDocument 413 /error/HTTP_REQUEST_ENTITY_TOO_LARGE.html.var # ErrorDocument 414 /error/HTTP_REQUEST_URI_TOO_LARGE.html.var # ErrorDocument 415 /error/HTTP_UNSUPPORTED_MEDIA_TYPE.html.var # ErrorDocument 500 /error/HTTP_INTERNAL_SERVER_ERROR.html.var # ErrorDocument 501 /error/HTTP_NOT_IMPLEMENTED.html.var # ErrorDocument 502 /error/HTTP_BAD_GATEWAY.html.var # ErrorDocument 503 /error/HTTP_SERVICE_UNAVAILABLE.html.var # ErrorDocument 506 /error/HTTP_VARIANT_ALSO_VARIES.html.var Here is the 404err.html file. I'm still not sure what to put in here. The first thought was to just use a blank white background, or a patterned background. For now its a jolly roger. Code: <html> <body bgcolor="707070"> <table width="100%" height="100%" align="center"> <tr> <td valign="middle" align="center"> <!-- <font face="Arial" color="crimson" size=1><b>Ads are blocked.</b></font> --> <img src="/m3ntal_com.gif"> <!-- <font face="Arial" color="crimson" size=1><b>Ads are blocked.</b></font> --> </td> </tr> </table> </body> </html> The results are as follows: Ultimately, even if using a simple solid colour blank page, I would prefer this on the router, where the 404 error on the router display's a simple page (or complicate one if you like). Cheers, Kev
Looks like this was a red herring. Just now I thought I saw the problem again but looks like the server I saw it on had a VPN connection open through which other DNS servers were available and that is how the ads are continuing to pop up on that box. When the VPN is disabled, the ads go away and show back up when the VPN is enabled. I just happened to notice it right around the time I started running DNSmasq 2.45 which caused me to suspect a connection but turned out to be the VPN I think. Sooo.... NEVER MIND!
Damwill: Not sure what u wanting to know exactly, This is my router with 7 machines using the net, download +/- 1mbyte/s, 4 wireless connections and 2nd router connected
So, I tried to disable the script by clearing the WAN Up script window in the GUI, saving it, then restarting dnsmasq, which worked...but, a few hours later, it started back up by itself it appears. When I SSH'd into the router, I found that the whole script is still sitting there in the script_wanup.sh. Do I need to delete that file and reboot the router?
I believe the script gets created at boot time. You can erase ir or reboot but shouldn't have to do both.
Would be nice to add a second file for custom blocking sides. I solved it for me with a second dnsmasq file that was load with the file from the script.
Hi, i just tried it out and work perfectly (tomato 1.21) What does this error mean: Jun 15 23:50:47 unknown daemon.crit dnsmasq[379]: failed to bind DHCP server socket: Address already in use Can i ignore it ? der_Kief
I've installed this script, just curious. When I go to "Digg.com" I am seeing adds tons more than if I enable my adblock extension. This just me, any one else able to test?
I use the MVPS list and see NO ads when I go to digg.com. I have not tried the method in this thread.
Hi there, i've just discovered this cool thread yesterday. I put the script and change the downloaded file from MVPS+yoyo to HSFS alone and it's a mess. Let me explain. In fact i often open many tabs at the same time and with HSFS or HFPS file it's very slow. All dns request goes to timeout and i have to refresh each tab manually to get my pages displayed.
Can someone tell me how to kill this script? I can't get globeinvestor.com to work, and I tried deleting the script, and reboot = no dice. Firmware reflash = no dice, ssh commands are ridiculously limited on tomato, so I can't do anything in there. Ads seem to be back now, but I still can't get to this site. I ran the script happily for over a month and a half, I got globeinvestor to work originally, but I had to reboot the router, and now it doesn't work anymore. I had globeandmail.com added to the whitelist, and that seemed to fix globeinvestor.com . If anyone could tell me how to get it working with the script it would be great, otherwise how do I get this thing off my router! Linksys WRT54GS v4, Tomato 1.21 Thanks.
I run this script and the site loads for me. Must be something with your ISP or on your PC If you have removed the script, saved then rebooted it would seem the problem is on your end and not the script.
http://www.theglobeandmail.com/ too? The site ran perfectly up until the script was originally put it, my ISP doesn't seem to block any other etrade sites, let alone anything else. Edit: Seems chrome isn't playing nicely with globeandmail, but IE and Firefox do. Thanks.
Has anyone been able to view episodes of heroes from nbc's website with this script? i noticed that i have to disable the script in order to watch webisodes on that site. anyone? thanks!
I do occasionally find sites or components sites I wish to view to get blocked by these lists, but you all do know the scripot has a whitelist section right?: Code: ## remove/whitelist websites ## removes 3 websites (aa.com, bb.com, cc.com) ## remove the # and edit the website urls. sed -i -e '/aa.com/d' $GENFILE sed -i -e '/bb.com/d' $GENFILE sed -i -e '/cc.com/d' $GENFILE Any time you have problems edit that section of the script with the site url you want to be exempted, save, then reboot. I think I have successfully added more white list lines on my script as well.
actually, i think it may have to do with the ads that nbc shows before they put on the webisode... the ads are coming from dblclick... i'm not excited about the idea of allowing dblclick thru the list tho. but AFTER the commercial plays, THEN it switches to nbc's content. i guess there's no way around this one huh?
Sounds like a reasonable explanation. I would try whitelisting it and try it out for say a week and see how many other doors whitelisting that opened. OR use a two front technique, white list them, but use something like adblock with /firefox to block their regular ads.
hoping ads from yahoo will be taken care of on your next version from this domain 'http://m1.2mdn.net'.
Thanks for the great script. Been using it for a while now. Don't see any bugs or any performance hit. Hope the new version is even better. (too bad I can't use this on my dd-wrt only router.)
I have just a little question: How can I add a domain to blacklist? For example, I want to blacklist 'www.microsoft.com', how can I do? Thanks.
Script not working... sorry if i write again (and for my bad english...) I'm try to use the script but i don't think it's working... If i try to surf / ping any site that should be blocked by the host-list i choosed (PGL YOYO), the router still can resolve the correct ip address. this is a portion of the generated /temp/hosts file: Code: address=/zde-affinity.edgecaching.ne/0.0.0.0 address=/zeads.com/0.0.0.0 address=/zedo.com/0.0.0.0 address=/zencudo.co.uk/0.0.0.0 address=/zenzuu.com/0.0.0.0 address=/zeus.developershed.com/0.0.0.0 address=/zintext.com/0.0.0.0 address=/zmedia.com/0.0.0.0 but if i try to do (in the ssh shell) ping Code: # ping zedo.com PING zedo.com (64.41.197.46): 56 data bytes 64 bytes from 64.41.197.46: icmp_seq=0 ttl=241 time=200.9 ms the logger says, when the script starts: Code: Jan 24 14:40:32 tomato user.notice root: ADBLOCK Load http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts Jan 24 14:40:32 tomato user.notice root: ADBLOCK Got Source Files Jan 24 14:40:32 tomato daemon.info dnsmasq[114]: exiting on receipt of SIGTERM Jan 24 14:40:33 tomato user.notice root: ADBLOCK Ignor Fail Safe Jan 24 14:40:40 tomato daemon.crit dnsmasq[391]: failed to bind DHCP server socket: Address already in use Jan 24 14:40:40 tomato daemon.crit dnsmasq[391]: FAILED to start up Jan 24 14:40:40 tomato user.notice root: ADBLOCK Ignor Fail Safe Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: started, version 2.41 cachesize 2048 Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: compile time options: no-IPv6 GNU-getopt no-RTC no-ISC-leasefile no-DBus no-I18N no-TFTP Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: asynchronous logging enabled, queue limit is 5 messages Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: DHCP, IP range 192.168.1.101 -- 192.168.1.149, lease time 1d Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: reading /etc/resolv.dnsmasq Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: using nameserver 208.67.220.220#53 Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: using nameserver 208.67.222.222#53 Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: read /etc/hosts - 0 addresses Jan 24 14:40:40 tomato daemon.info dnsmasq[393]: read /etc/hosts.dnsmasq - 3 addresses Jan 24 14:40:40 tomato user.notice root: ADBLOCK Unique Hosts Blocked 2549 I'm running Tomato SpeedMod Firmware v1.19.8502 Thank in advance
Jan 24 14:40:40 tomato user.notice root: ADBLOCK Unique Hosts Blocked 2549 Doesn't seem that it fully works. Mine is : 16160 but i have 2 files to dl : Jan 24 13:56:35 WRT user.notice root: ADBLOCK Downloaded http://www.mvps.org/winhelp2002/hosts.txt Jan 24 13:56:35 WRT user.notice root: ADBLOCK Load http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts Try to add the second one and reboot.
Tanks for your answer. I've also tried by adding the mvps host-list, but there is no difference, the router can still resolve (and ping) the blocked domains.
Seems to be the same for me as i tested after i saw your question. Sounds strange because it loads the files correcty. Jan 25 21:06:26 WHR54HP user.notice root: ADBLOCK Unique Hosts Blocked 16160 I've just change few parameters since i went back to my Buffalo. I've unchecked I didn't try if it's that.
I'm heartbroken that I haven't been able to view the NBC videos also. I tried whitelisting doubleclick.net/doubleclick.com but I just can't seem ever to get past the "Connecting..." message on the NBC videos. I really miss not seeing the Fey/Palin SNL skits. But I love the safety that this Adblock script gives my network. Can anyone spell out for me/us just how we can view the NBC videos, while using this great network script? Thanks, in advance!
Thanks for your reply. I'm now at work so i can't be sure, but I think I have the 'Intercept DNS Port' checked. I will try by unchecking. But this way, people in the network could use a DNS server other then the router one?
