############################################## * ALL-U-NEED Adblocking by Yaqui * DNS Cache Poisoning Edition ############################################## Description: ======================= I created this for those who favor the "Dnsmasq way" (The host method is still also posted under the Dnsmasq script) Main Features: ===================== ** I encapsulated it in it's own script rather than having to cron schedule the main wan-up script. (Useful if you wish to do other things in wan-up without having to have them scheduled.) ++ Easy-Edit Whitelist Sites ++ Modular - Function Code, Easy to Maintain ++ Failsafe Test - Web access check without errors ++ List/Data Format Test ++ Data Population Test ++ Entry Number Report by Fast wc method ++ Multiple Data Sources - Easy to edit ++ Condensed/shorter Changelog: ===================== v2.92 Further variable condensing, ADBLOCK.sh is now only 2,989 bytes! v2.91 Out of memory error fix, added loop for whitelist:can enter more sites v2.9e Temp file was not appending correctly during cleanup v2.9d Changed flag location, log now reports cleanups, better error reports v2.9c Minor updates - download flag w/cleanup function, added cron reference v2.9 More sed updates - some url/ip dots were not being removed v2.8 Minor bug fix with sedfltr seds v2.7 Better Memory Management - Delete Duplicates after each download. v2.5 Better Failsafe Test and Cleardata Function v2.4 Added AutoUpdate Option v2.3 Added Whitelist, UseWhitelist, Optimise options to Variable Section v2.0 MAJOR Code re-write using Functions, Now has 4 sources, *much* easier to maintain v1.5 Allow for changing sources: edit SOURCE1 and SOURCE2 variables v1.4 Log now reports number of entries using ultra-fast wc method Extra SED statement added: [0-9]www[0-9] was not being removed v1.3 Failsafe Test Method Update v1.2 Condensed Further, no more need for ALL data to be present (you will still get some Adblocking if a list was not downloaded for some reason) minor bug fixes v1.1 Added Failsafe Test & Data Population/Formatting Test v1.0 Needed Sed Statement Corrections - Thank you to testers Thanks and credits go to other authors for inspiration. ** First: (If you are NOT Using the Optimise Option In the Script) Paste this under Advanced > DHCP / DNS > Dnsmasq Custom Configuration box in Tomato ; Then SAVE. Code: cache-size=2048 log-async=5 dhcp-authoritative ** Next: Place this script under Administration > Scripts > WAN Up tab ; then SAVE ; then Reboot. Code: ## ALL-U-NEED AdBlocking By YAQUI 5/1/09 ## Dnsmasq Ed. v2.92 sleep 20 ## Auto Update? (Y or N) AUPD="N" ## Create ADBLOCK.sh rm -f /tmp/ADBLOCK.sh ADB="/tmp/ADBLOCK.sh" touch $ADB ( cat <<'ENDF' #!/bin/sh ## EDITABLE VARIABLES OPTIMISE="N" S1="http://www.mvps.org/winhelp2002/hosts.txt" # ~612K S2="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" # ~72K S3="http://someonewhocares.org/hosts/hosts" # ~208K S4="http://hostsfile.mine.nu/Hosts" # ~2.59M GETS1="Y" GETS2="Y" GETS3="N" GETS4="N" USEWHITELIST="Y" # Enter sites in format below WHITE="editme.com editme.com editme.com" ## DO NOT EDIT BELOW NIP="0.0.0.0" GEN="/tmp/gen" TMP="/tmp/temp" D="" ## FUNCTIONS CLR () { rm -f $GEN rm -f $TMP touch $GEN touch $TMP } FMEM () { service dnsmasq stop killall -9 dnsmasq logger ADBLOCK Freeing Memory } DS1 () { ##(Inline grep for 127.0.0.1 & Delete DOS Format Returns) D=0 if [[ $GETS1 == "Y" ]] ; then if wget $S1 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved S1 $S1 D=1 else logger ADBLOCK S1 ERROR fi fi } DS2 () { D=0 if [[ $GETS2 == "Y" ]] ; then if wget $S2 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved S2 $S2 D=1 else logger ADBLOCK S2 ERROR fi fi } DS3 () { D=0 if [[ $GETS3 == "Y" ]] ; then if wget $S3 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved S3 $S3 D=1 else logger ADBLOCK S3 ERROR fi fi } DS4 () { D=0 if [[ $GETS4 == "Y" ]] ; then if wget $S4 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved S4 $S4 D=1 else logger ADBLOCK S4 ERROR fi fi } CLN () { if [[ $D = 1 ]] ; then sed -i -e 's/[[:cntrl:][:blank:]]//g' $GEN sed -i -e '/\#.*$/ s/\#.*$//' $GEN sed -i -e '/\[.*\]/ s/\[.*\]//' $GEN sed -i -e '/^$/d' $GEN sed -i -e '/127\.0\.0\.1/ s/127\.0\.0\.1//' $GEN sed -i -e '/^www[0-9]*\./ s/^www[0-9]*\.//' $GEN sed -i -e '/^[0-9]*www[0-9]*\./ s/^[0-9]*www[0-9]*\.//' $GEN sed -i -e '/^www\./ s/^www\.//' $GEN sed -i -e '/</d' $GEN sed -i -e 's/^[ \t]*//;s/[ \t]*$//' $GEN cat $GEN | sort -u > $TMP mv $TMP $GEN rm -f $TMP logger ADBLOCK List Cleaned fi } FDNSM () { sed -i -e 's|$|/'$NIP'|' $GEN sed -i -e 's|^|address=/|' $GEN } LCFG () { cat /etc/dnsmasq.conf >> $GEN } OPT () { if [[ $OPTIMISE == "Y" ]] ; then cat >> $GEN <<EOF cache-size=2048 log-async=5 EOF fi } LWHT () { if [[ $USEWHITELIST == "Y" ]] ; then for site in $WHITE do sed -i -e "/$site/d" $GEN done logger ADBLOCK Whitelist Applied fi } LBLK () { dnsmasq --conf-file=$GEN } TST () { sleep 15 if sed -n -e '/^address=\/ad\..*\..*\/0\.0\.0\.0$/p' $GEN ; then TOT=`wc -l $GEN | cut -d" " -f5` logger ADBLOCK List Sample Format SUCCESS logger ADBLOCK List Contains End Total of $TOT Entries else logger ADBLOCK List ERROR fi } FS () { if ps | grep -E "dnsmasq" | grep -E "nobody" ; then logger ADBLOCK Dnsmasq is Running Failsafe Ignored else logger ADBLOCK Dnsmasq NOT Running Dnsmasq Restarting service dnsmasq stop killall -9 dnsmasq dnsmasq fi } ## Run Functions CLR FMEM DS1 CLN DS2 CLN DS3 CLN DS4 CLN FDNSM LCFG OPT LWHT LBLK TST FS CLR ## End of ADBLOCK.sh ENDF ) > $ADB AUP () { if [[ $AUPD == "Y" ]] ; then if [[ $(cru l | grep AdUpd | cut -d "#" -f2) != "AdUpd" ]] ; then ## cru (a)dd <name> "min hr day mo wkday <cmd>" ## min=0-59 hour=0-23 day=1-31 month=1-12 sun=0 *=all cru a AdUpd "0 0 * * 2 $ADB" fi fi } ## Run ADBLOCK.sh & AUP chmod 777 $ADB $ADB AUP ############################################## * Quick Adblock Disable Method * ############################################## Someone wanted a quick way to disable adblocking using the SES Button. So here it is: 1) Place this script under Administration > Buttons / LED in custom box 2) Use one set of seconds for this custom script 3) Then just use another set of seconds and select the reboot choice for it; to get adblocking back. Code: ## SES - DISABLE Adblocking by Yaqui v1.0 if ps | grep tmp/gen ; then service dnsmasq stop killall -9 dnsmasq logger ADBLOCK SES button activated adblock shutdown sleep 2 dnsmasq led amber on #Turn on warning light that adblock is off! fi ############################################## ALL-U-NEED Adblocking by Yaqui * Host Edtion * ############################################## Description: =================== This script uses the Host Method of blocking ads. Some people prefer this over the above Dnsmasq Method. Changelog: =================== v2.0c Sed www fix v2.0b Delete duplicates fixed v2.0a Minor duplicate entry fix v2.0 Fixed many bugs, sed formats and temp appending v1.5 Code re-write using Functions and more Sources v1.0 First version, have not found problems yet Code: ## ALL-U-NEED Ad Blocking By YAQUI 02/15/2009 ## Host Method Edition v2.0c sleep 20 ## Auto Update? (Y or N) AUPD="N" ## Create ADBLOCK.sh ADBLK="/tmp/ADBLOCK.sh" touch $ADBLK ( cat <<'ENDF' #!/bin/sh ## EDITABLE VARIABLES SRC1="http://www.mvps.org/winhelp2002/hosts.txt" # ~612K SRC2="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" # ~72K SRC3="http://someonewhocares.org/hosts/hosts" # ~208K SRC4="http://hostsfile.mine.nu/Hosts" # ~2.59M GETSRC1="Y" GETSRC2="Y" GETSRC3="N" GETSRC4="N" USEWHITELIST="N" WHITE1="whitesite1.com" WHITE2="whitesite2.com" WHITE3="whitesite3.com" WHITE4="whitesite4.com" WHITE5="whitesite5.com" ## DO NOT EDIT BELOW NULLIP="0.0.0.0" GEN="/tmp/hosts" TEMP="/tmp/temp" HOSTS="/etc/hosts" D="" ## FUNCTIONS CLRDATA () { rm -f $GEN rm -f $TEMP } DLSRC1 () { ##(Inline grep for 127.0.0.1 & Delete DOS Format Returns) D=0 if [[ $GETSRC1 == "Y" ]] ; then if wget $SRC1 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC1 $SRC1 D=1 else logger ADBLOCK SRC1 ERROR fi fi } DLSRC2 () { D=0 if [[ $GETSRC2 == "Y" ]] ; then if wget $SRC2 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC2 $SRC2 D=1 else logger ADBLOCK SRC2 ERROR fi fi } DLSRC3 () { D=0 if [[ $GETSRC3 == "Y" ]] ; then if wget $SRC3 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC3 $SRC3 D=1 else logger ADBLOCK SRC3 ERROR fi fi } DLSRC4 () { D=0 if [[ $GETSRC4 == "Y" ]] ; then if wget $SRC4 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC4 $SRC4 D=1 else logger ADBLOCK SRC4 ERROR fi fi } CLEANUP () { if [[ $D = 1 ]] ; then sed -i -e 's/[[:cntrl:][:blank:]]//g' $GEN sed -i -e '/\#.*$/ s/\#.*$//' $GEN sed -i -e '/\[.*\]/ s/\[.*\]//' $GEN sed -i -e '/^$/d' $GEN sed -i -e '/127\.0\.0\.1/ s/127\.0\.0\.1//' $GEN #sed -i -e '/^www[0-9]*\./ s/^www[0-9]*\.//' $GEN #sed -i -e '/^[0-9]*www[0-9]*\./ s/^[0-9]*www[0-9]*\.//' $GEN #sed -i -e '/^www\./ s/^www\.//' $GEN sed -i -e '/</d' $GEN sed -i -e 's/^[ \t]*//;s/[ \t]*$//' $GEN sed -i -e '/localhost/d' $GEN logger ADBLOCK List Cleaned fi } DELDUPES () { if [[ $D = 1 ]] ; then touch $TEMP touch $GEN cat $GEN | sort -u > $TEMP mv $TEMP $GEN rm -f $TEMP fi } FMTHOST () { cat $GEN >> $HOSTS sed -i -e 's|^|'$NULLIP' |' $HOSTS sed -i -e '1i127.0.0.1 localhost' $HOSTS } LDWLIST () { if [[ $USEWHITELIST == "Y" ]] ; then sed -i -e "/$WHITE1/d" $HOSTS sed -i -e "/$WHITE2/d" $HOSTS sed -i -e "/$WHITE3/d" $HOSTS sed -i -e "/$WHITE4/d" $HOSTS sed -i -e "/$WHITE5/d" $HOSTS fi } LOADHOSTS () { killall -1 dnsmasq logger ADBLOCK Restarting Dnsmasq } TESTPOPFMT () { sleep 15 if sed -n -e '/^0\.0\.0\.0.ads./p' $HOSTS ; then ENDTOT=`wc -l $HOSTS | cut -d" " -f5` logger ADBLOCK Data List Population and Format SUCCESS logger ADBLOCK Data List Contains End Total of $ENDTOT Entries else logger ADBLOCK List ERROR fi } FAILSAFE () { if ps | grep -E "dnsmasq" | grep -E "nobody" ; then logger ADBLOCK Dnsmasq is Running Failsafe Ignored else logger ADBLOCK Dnsmasq NOT Running Dnsmasq Restarting service dnsmasq stop killall -9 dnsmasq dnsmasq fi } ## Run Functions CLRDATA DLSRC1 CLEANUP DLSRC2 CLEANUP DELDUPES DLSRC3 CLEANUP DELDUPES DLSRC4 CLEANUP DELDUPES FMTHOST LDWLIST LOADHOSTS TESTPOPFMT FAILSAFE CLRDATA ## End of ADBLOCK.sh ENDF ) > $ADBLK AUTOUP () { if [[ $AUPD == "Y" ]] ; then if [[ $(cru l | grep AdUpd | cut -d "#" -f2) != "AdUpd" ]] ; then ## cru (a)dd <name> "min hr day mo wkday <cmd>" ## min=0-59 hour=0-23 day=1-31 month=1-12 sun=0 *=all cru a AdUpd "0 0 * * 2 $ADBLK" fi fi } ## Run ADBLOCK.sh & AUTOUP chmod 777 $ADBLK $ADBLK AUTOUP
Why have folk gone back to hosts site blocking? I'm still using the dnsmasq domain blocking as it seems much more efficient, smaller files, quicker lookup. http://www.linksysinfo.org/forums/showthread.php?t=53904 If the issue is that the mvps list is better - cannot it be processed into dnsmasq domain format by stripping off the subdomains and sorting uniquely?
Cause everyone wants to do it their own way, which really is fine. I'm using a variation of the same code I wrote more than a year ago. (Unchanged since August 07.) Still works great... Why change it? Also, I haven't seen any real data that shows blocking via a dnsmasq config file is more efficient than having dnsmasq process a hosts file. Where is it shown that for the same number of hosts... 1) one uses less memory than another? (filesize and dnsmasq usage combined...) 2) one allows for faster lookups than another? (Any way to test this on a router? Doubt it...) Please point me to the hard data...
Interpreting Linux memory usage figures is a dark art!, but easy to see the ram disk usage based on filesize - although I store my manually updated/edited dnsmasq config on /jffs 75 host entries for doubleclick.net need only 1 'cached' dnsmaq domain config entry, plus any new subdomains get automatically blocked.
same here: Jan 1 01:02:43 WL500GP user.notice root: WAN UP Script will execute after 70sec please wait..... Jan 27 09:21:22 WL500GP cron.err crond[130]: time disparity of 20550739 minutes detected Jan 27 09:22:18 WL500GP user.notice root: Adblock Downloaded http://www.mvps.org/winhelp2002/hosts.txt Jan 27 09:22:18 WL500GP user.notice root: Adblock Load http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts Jan 27 09:22:18 WL500GP user.notice root: Adblock Received Source Data Jan 27 09:22:19 WL500GP daemon.info dnsmasq[173]: exiting on receipt of SIGTERM Jan 27 09:22:19 WL500GP user.notice root: Adblock Ignore Failsafe Jan 27 09:22:30 WL500GP daemon.crit dnsmasq[428]: error at line 1 of /tmp/hosts Jan 27 09:22:30 WL500GP daemon.crit dnsmasq[428]: FAILED to start up Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: started, version 2.46 cachesize 150 Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N no-TFTP Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: DHCP, IP range 192.168.1.100 -- 192.168.1.149, lease time 1d Jan 27 09:22:30 WL500GP user.notice root: Adblock Failsafe Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: reading /etc/resolv.dnsmasq Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: using nameserver 213.33.98.136#53 Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: using nameserver 195.3.96.67#53 Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: read /etc/hosts - 0 addresses Jan 27 09:22:30 WL500GP daemon.info dnsmasq[431]: read /etc/hosts.dnsmasq - 1 addresses
Hi, after using xcooling's adblock script for a while without any troubles i just give your script a try. Comparing each other i would say there is no big difference but internet seems to be a bit more responsive with your script. Also its more reasonable for me der_Kief
Which one will work if i plug the router 1 (with internet connection) to a lan port of my tomato's one ? Host better than DNS ?
I enabled OPTIMIZE and here is my log: Code: Feb 8 23:48:14 ? user.notice root: ADBLOCK Free Memory Stage Stopped Dnsmasq Feb 8 23:48:23 ? user.notice root: ADBLOCK Retrieved SRC1 http://www.mvps.org/winhelp2002/hosts.txt Feb 8 23:48:44 ? user.notice root: ADBLOCK Retrieved SRC2 http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts Feb 8 23:49:03 ? daemon.crit dnsmasq[685]: illegal repeated keyword at line 16135 of /tmp/hosts Feb 8 23:49:03 ? daemon.crit dnsmasq[685]: FAILED to start up Feb 8 23:49:19 ? user.notice root: ADBLOCK Data List Population and Format SUCCESS Feb 8 23:49:19 ? user.notice root: ADBLOCK Data List Contains End Total of 16136 Entries Feb 8 23:49:19 ? user.notice root: ADBLOCK Dnsmasq not Running Dnsmasq Restart yup... daemon.crit dnsmasq[685]: FAILED to start up...
Hi, i also have optimization ON but there are no problems at all. Did you try the newest version ? Maybe something went wrong at downloading the host files ! Do use a tomato mod ? Here is my log: Code: daemon.info dnsmasq[138]: exiting on receipt of SIGTERM user.notice root: ADBLOCK Free Memory Stage Stopped Dnsmasq user.notice root: ADBLOCK Retrieved SRC1 http://www.mvps.org/winhelp2002/hosts.txt user.notice root: ADBLOCK Retrieved SRC2 http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts daemon.info dnsmasq[498]: started, version 2.46 cachesize 1024 daemon.info dnsmasq[498]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N no-TFTP daemon.info dnsmasq[498]: asynchronous logging enabled, queue limit is 5 messages daemon.info dnsmasq[498]: reading /etc/resolv.dnsmasq daemon.info dnsmasq[498]: using nameserver xxx.xxx.xxx.xxx#53 daemon.info dnsmasq[498]: using nameserver xxx.xxx.xxx.xxx#53 daemon.info dnsmasq[498]: read /etc/hosts - 0 addresses daemon.info dnsmasq[498]: read /etc/hosts.dnsmasq - 1 addresses user.notice root: ADBLOCK Data List Population and Format SUCCESS user.notice root: ADBLOCK Data List Contains End Total of 16124 Entries user.notice root: ADBLOCK Dnsmasq is Running Failsafe Ignored der_Kief
I use 1.23 VPN GUI Mod. I posted my message after trying this adblock (ver 2.9c). I tried again (before this post), but dnsmasq can't start with "optimise" enabled. edit: err... ok, the VPN GUI Mod seems to have this by default: Code: cache-size=2048 log-async=5 and it was appended at the end of /tmp/hosts... edit2: it works if I remove it, but dnsmasq is started with a cache size of 150 instead of 2048... I don't understand ô_O edit3: ok... I put back my settings and now I only have one entry in /tmp/hosts, when a minute ago I had duplicate entry -_-; so now it works fine like how it should
Which script are you using? The Hosts or Cache poisoning method? Am I correctly assuming the host method script goes under admin-scripts.asp -> WAN UP
Hi, i'm using the DNS Cache Poisoning Edition. Both scripts should be placed in the WAN-up section. der_Kief
Got probs with the host method. Feb 12 22:26:05 Cougar daemon.err dnsmasq[561]: bad address at /etc/hosts line 16825 Feb 12 22:26:05 Cougar daemon.err dnsmasq[561]: bad address at /etc/hosts line 16826 Feb 12 22:26:05 Cougar daemon.err dnsmasq[561]: bad address at /etc/hosts line 16827 .......
I only use the MVPS hosts file Use SSH to check out the hosts file being generated - sounds like there may be some special characters being loaded into the hosts file causing your problems Copy and paste the three lines in question 16825,16826 and 16827 - I'd be interested to see what's happening
Too late, when i had that problem i switched to dns poisong version and it wasn't just 3 lines, i couldn't read my log because it was full of such line for each one of them.
After a clean full erase i have the same here is parts of my hosts file : 0.0.0.0102.122.2o7.net 0.0.0.0102.112.2o7.net 0.0.0.0102.112.207.net 0.0.0.0101order.com 0.0.0.0101com.com 0.0.0.010168.hittail.com 0.0.0.0100webads.com 0.0.0.010016.searchmiracle.com 0.0.0.010006.hittail.com 0.0.0.010000hits.net 0.0.0.0100.topnews.ru There's no space between, is it normal ? I don't think so as in my hosts.dnsmasq there is. I guess it could be that.
@Peyton Thanks for posting the results, i'll look into it and fix it. probably just one of the sed statements is off a bit.
Host Edition should be working fine now: Sample /etc/hosts format: Code: 127.0.0.1 localhost 0.0.0.0 0.r.msn.com 0.0.0.0 000-search.net 0.0.0.0 000dom.revenuedirect.com 0.0.0.0 00119922.com 0.0.0.0 008.free-counter.co.uk 0.0.0.0 00fun.com 0.0.0.0 011707160008.c.mystat-in.net 0.0.0.0 032439.com 0.0.0.0 061606084448.c.mystat-in.net 0.0.0.0 070806142521.c.mystat-in.net 0.0.0.0 08search.com 0.0.0.0 090906042103.c.mystat-in.net 0.0.0.0 092706152958.c.mystat-in.net 0.0.0.0 093qpeuqpmz6ebfa.com 0.0.0.0 098765.com 0.0.0.0 0fficial-page.com 0.0.0.0 0ml.net 0.0.0.0 0scanner.com 0.0.0.0 0texkax7c6hzuidk.com 0.0.0.0 1-se.com 0.0.0.0 1.9797aiai.com 0.0.0.0 1.adbrite.com Log Sample Output: Code: Feb 15 13:53:07 user.notice root: ADBLOCK Retrieved SRC1 http://www.mvps.org/winhelp2002/hosts.txt Feb 15 13:53:25 user.notice root: ADBLOCK List Cleaned Feb 15 13:53:28 user.notice root: ADBLOCK Retrieved SRC2 http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts Feb 15 13:53:30 user.notice root: ADBLOCK List Cleaned Feb 15 13:53:42 user.notice root: ADBLOCK Restarting Dnsmasq Feb 15 13:53:44 daemon.info dnsmasq[146]: read /etc/hosts - 17140 addresses Feb 15 13:53:44 daemon.info dnsmasq[146]: read /etc/hosts.dnsmasq - 1 addresses Feb 15 13:53:58 user.notice root: ADBLOCK Data List Population and Format SUCCESS Feb 15 13:53:58 user.notice root: ADBLOCK Data List Contains End Total of 17140 Entries Feb 15 13:54:14 user.notice root: ADBLOCK Dnsmasq is Running Failsafe Ignored
2.9c or 2.9d was working fine, but when I updated to latest version 2.9e, its not working except if I leave all as default (meaning no changes to script). I'll keep checking to see why.
Hi, after upgrading to 2.9e i get the following error message: Feb 15 21:22:19 TOMATO user.err kernel: Out of Memory: Killed process 484 (dnsmasq). Before (with 2.9c) everything was working perfect ! Hope yaqui will fix this soon der_Kief
When I add a site to the whitelist, it does not seem to work. Example; USEWHITELIST="Y" W1="www.theglobeandmail.com" W2="whitesite2.com" W3="whitesite3.com" W4="whitesite4.com" W5="whitesite5.com" When I try to access the above globeandmail site, it blocks it. It was working before with 2.9c.
Well, no problems here but I have rather strange request: Since the ads are being blocked - IE and Firefox both show 404 pages instead of banners (which is great) - i wonder is it possible to replace 404 page with something like plain white html or even transparent gif image for the sake of aestethics. Just an idea... Cheers!
You say you have no issues, can you test if you can access this webpage? http://www.theglobeandmail.com/ I put this site on the whitelist, but its still not working. If its working for you, then the Yaqui's script is fine, and its something else in my setup. I'd like to revert back to the former 2.9c script, but I did not make a copy of it.
Works fine here: ... EDIT1: WOOPS, seem's that i'm running older version - well it works here... EDIT2: after update, still works... using first two hosts files in config. Check in access restriction, maybe you've put it in by mistake
Hmm not so cool... I notice that when running Dnsmasq Ed v2.9e my WRT54GL gets low on free memory. I've seen it go as low as 484 KB with just me as user. How much free memory do you and other users have left?
it's sad that YAQUI doesn't read often the forum, I may switch back to the previous one I was using... Code: ## ALL-U-NEED Ad Blocking By YAQUI 02/08/2009 ## Dnsmasq Edition v2.9c sleep 20 ## Auto Update? (Y or N) AUTOUPDATE="Y" ## Create ADBLOCK.sh ADBLK="/tmp/ADBLOCK.sh" touch $ADBLK ( cat <<'ENDF' #!/bin/sh ## EDITABLE VARIABLES USEWHITELIST="Y" WHITE1="feedburner.com" WHITE2="whitesite2.com" WHITE3="whitesite3.com" OPTIMISE="N" GETSRC1="Y" GETSRC2="Y" GETSRC3="N" GETSRC4="N" SRC1="http://www.mvps.org/winhelp2002/hosts.txt" # ~612K SRC2="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" # ~72K SRC3="http://someonewhocares.org/hosts/hosts" # ~208K SRC4="http://hostsfile.mine.nu/Hosts" # ~2.59M ## DO NOT EDIT BELOW NULLIP="0.0.0.0" GEN="/tmp/hosts" TEMP="/tmp/temp" D="" ## FUNCTIONS CLRDATA () { rm -f $GEN touch $GEN } FREEMEM () { service dnsmasq stop killall -9 dnsmasq logger ADBLOCK Free Memory Stage Stopped Dnsmasq } DLSRC1 () { ##(Inline grep for 127.0.0.1 & Delete DOS Format Returns) if [[ $GETSRC1 == "Y" ]] ; then if wget $SRC1 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC1 $SRC1 D=1 fi else D=0 fi } DLSRC2 () { if [[ $GETSRC2 == "Y" ]] ; then if wget $SRC2 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC2 $SRC2 D=1 fi else D=0 fi } DLSRC3 () { if [[ $GETSRC3 == "Y" ]] ; then if wget $SRC3 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC3 $SRC3 D=1 fi else D=0 fi } DLSRC4 () { if [[ $GETSRC4 == "Y" ]] ; then if wget $SRC4 -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved SRC4 $SRC4 D=1 fi else D=0 fi } CLEANUP () { if [[ $D = 1 ]] ; then sed -i -e 's/[[:cntrl:][:blank:]]//g' $GEN sed -i -e '/\#.*$/ s/\#.*$//' $GEN sed -i -e '/\[.*\]/ s/\[.*\]//' $GEN sed -i -e '/^$/d' $GEN sed -i -e '/127\.0\.0\.1/ s/127\.0\.0\.1//' $GEN sed -i -e '/^www[0-9]*\./ s/^www[0-9]*\.//' $GEN sed -i -e '/^[0-9]*www[0-9]*\./ s/^[0-9]*www[0-9]*\.//' $GEN sed -i -e '/^www\./ s/^www\.//' $GEN sed -i -e '/</d' $GEN sed -i -e 's/^[ \t]*//;s/[ \t]*$//' $GEN touch $TEMP cat $GEN | sort -u > $TEMP mv $TEMP $GEN rm -f $TEMP fi } FMTDNSM () { sed -i -e 's|$|/'$NULLIP'|' $GEN sed -i -e 's|^|address=/|' $GEN } LDDNSMCFG () { cat /etc/dnsmasq.conf >> $GEN } OPTDNSM () { if [[ $OPTIMISE == "Y" ]] ; then cat >> $GEN <<EOF cache-size=2048 log-async=5 EOF fi } LDWLIST () { if [[ $USEWHITELIST == "Y" ]] ; then sed -i -e "/$WHITE1/d" $GEN sed -i -e "/$WHITE2/d" $GEN sed -i -e "/$WHITE3/d" $GEN fi } LDBLIST () { dnsmasq --conf-file=$GEN } TESTPOPFMT () { sleep 15 if sed -n -e '/^address=\/ad\..*\..*\/0\.0\.0\.0$/p' $GEN ; then endtotal=`wc -l /tmp/hosts | cut -d" " -f5` logger ADBLOCK Data List Population and Format SUCCESS logger ADBLOCK Data List Contains End Total of $endtotal Entries else logger ADBLOCK List ERROR fi } FAILSAFE () { if ps | grep -E "dnsmasq" | grep -E "nobody" ; then logger ADBLOCK Dnsmasq is Running Failsafe Ignored else logger ADBLOCK Dnsmasq not Running Dnsmasq Restart service dnsmasq stop killall -9 dnsmasq dnsmasq fi } ## Run Functions CLRDATA FREEMEM DLSRC1 CLEANUP DLSRC2 CLEANUP DLSRC3 CLEANUP DLSRC4 CLEANUP FMTDNSM LDDNSMCFG OPTDNSM LDWLIST LDBLIST TESTPOPFMT FAILSAFE #CLRDATA ## End of ADBLOCK.sh ENDF ) > $ADBLK AUTOUPDATER () { if [[ $AUTOUPDATE == "Y" ]] ; then if [[ $(cru l | grep AdlistUpd | cut -d "#" -f2) != "AdlistUpd" ]] ; then ## cru (a)dd <name> "min hr day mo wkday <cmd>" min=0-59 hour=0-23 day=1-31 month=1-12 sun=0 *=all cru a AdlistUpd "0 4 * * 2 $ADBLK" fi fi } ## Run ADBLOCK.sh & AUTOUPDATER chmod 777 $ADBLK $ADBLK AUTOUPDATER
Thanks srouquette for bring back v2.9c. I think YAQUI is busy at the moment so it may need some time till he fix this. Alternative people can us xcooling's script. der_Kief
I noticed that xcooling's script isn't such a memory hog as YAQUI's... (I tried both 2.9c and 2.9e) For the moment xcooling's script has my preference, free memory with mvps & yoyo lists is about 3,716.00 KB (25.64%).
What kind of router do you have ? My WRT54GL has about 1300 KB of free memory with xcooling's script and mvps&yoyo lists. Maybe you have enabled "Count cache memory as free memory" under the administration->debugging section !? der_Kief
I've got a WRT54GL too and it's currently running Tomato 1.23ND. With xcooling's script I now have: Free Memory 3,184.00 KB (21.97%) while using mvps & yoyo lists. Count cache memory as free memory is disabled.
I read the above and switched from Yaqui's script to xcooling's script, but i find that I can still ping the addresses from the mvps or yoyo lists. Using Yaqui's script, the addresses are not pingable. Am I right? Does anyone know why? It seems to me that Yaqui's script is working better than xcooling's script, with the exception of the whitelist not working properly in Yaqui's script.
Sorry my mistake. xcooling's script works fine, and it was my mistake after tweaking the script! I've reverted back to xcooling's script.
Ok, I give up. Both xcooling's script and Yaqui's script, when you enable dnsmasq optimization, they both don't allow you to go to the www.theglobeandmail.com website when you add www.theglobeandmail.com into the whitelist.
optimization adds some lines in the dnsmasq conf, maybe you already have these lines on your custom dns settings (log size and cache).
Yep. I originally had those optimization lines in my custom dns settings. So, I removed it and enabled the Optimize DNS in both Yaqui's and xcooling's scripts, but they did not allow me to go to my whitelisted websites. Next, I also disabled the Optimize DNS in the scripts, and added back the optimization lines via my custom dns settings, and same result. Not sure if its caching the settings someplace else. I re-booted my router each time after every change. I am sure it worked before. I'm not sure if rebooting the router is sufficient after each change. I'll continue with xcooling's script for now.
The globeandmail site works for me using the MVPS source. It's [BOLD]very[/BOLD] javascript heavy which may also be part of your problems - site is slow for me to load
Thanks mikester. I am not sure if there is a difference but the whitelist indicates aa.com or bb.com as examples, and when I used "www.theglobeandmail.com", it did not work to connect to the website. But when I used "theglobeandmail.com", it worked! Anyways, its working so far for the whitelist.
After my mistake with entering the whitelist sites, I re-tried Yaqui's script and used just "theglobeandmail.com" and Yaqui's v2.9e script is working fine. So, it was my error in adding the format for whitelist sites. I was using "www.yyy.com" instead of "yyy.com" for the whitelist sites. Is there any pros and cons with Yaqui's and xcooling's script? Which one is better?
Yagui, great script! Is it posibble that a SES button script could be created to toggle the ad blocking on and off? This would be handy, as sometimes it would be nice to temporarily disable what is blocked so I can get through something that requires referral clicks for example. Just a suggestion, thanks!
Fixed I fixed the dns edition script and added optimisations, along with a loop for the whitelist so you can enter more sites without having to mess with additional sed statements. Sorry it took awhile, been busy with crap at home. ADBLOCK.sh is also smaller now ... only 2,989 bytes. It works with S1 and S2, I don't have memory for all 4 sources, if someone does... please test. Code: May 1 12:42:51 daemon.info dnsmasq[143]: exiting on receipt of SIGTERM May 1 12:42:51 user.notice root: ADBLOCK Freeing Memory May 1 12:42:55 user.notice root: ADBLOCK Retrieved S1 http://www.mvps.org/winhelp2002/hosts.txt May 1 12:43:13 user.notice root: ADBLOCK List Cleaned May 1 12:43:15 user.notice root: ADBLOCK Retrieved S2 http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts May 1 12:43:27 user.notice root: ADBLOCK List Cleaned May 1 12:43:33 user.notice root: ADBLOCK Whitelist Applied May 1 12:43:51 user.notice root: ADBLOCK List Sample Format SUCCESS May 1 12:43:51 user.notice root: ADBLOCK List Contains End Total of 15905 Entries May 1 12:43:51 user.notice root: ADBLOCK Dnsmasq is Running Failsafe Ignored
I haven't tested that version put with the previous one i can't reach some website like download center from microsoft just after i put any windows version and want to download them manually. http://www.microsoft.com/downloads/...eriod=&sortCriteria=date&nr=30&DisplayLang=fr
You could do something like this: 1) Place this script under Administration > Buttons / LED in custom box 2) Use one set of seconds for this custom script, 3) Then just use another set of seconds and select the reboot choice to get adblocking back. Code: ## SES - DISABLE Adblocking by Yaqui v1.0 if ps | grep tmp/gen ; then service dnsmasq stop killall -9 dnsmasq logger ADBLOCK SES button activated adblock shutdown sleep 2 dnsmasq # Turn on warning light that adblock is off! led amber on fi Unfortunately, there seems to be no easy way to just unload the dnsmasq.conf file or reload without restarting dnsmasq.
are there any lists available that corral all the chinese adservers ? I have a site which mainly has Chinese students with corresponding browsing of chinese websites.
Quick question.....How do I edit the whitelist? I see this section USEWHITELIST="Y" # Enter sites in format below WHITE="editme.com editme.com editme.com" So if I waned to add a few sites would I do something like this? USEWHITELIST="Y" # Enter sites in format below WHITE="hulu.com somesite.com thebestsiteintheworld.com" I see this thread over here http://www.linksysinfo.org/forums/showthread.php?t=57556 Whats the differences? Does one have an advantage over the other?
Autoupdate don't work for me. No idea why. Nothing in logs. Code: ## Auto Update? (Y or N) AUPD="Y" Code: AUP () { if [[ $AUPD == "Y" ]] ; then if [[ $(cru l | grep AdUpd | cut -d "#" -f2) != "AdUpd" ]] ; then ## cru (a)dd <name> "min hr day mo wkday <cmd>" ## min=0-59 hour=0-23 day=1-31 month=1-12 sun=0 *=all cru a AdUpd "5 3 * * 0 $ADB" fi fi }
Is it possible to make a blacklist as well, to add URLs that aren't included in any of the sites in the script? Or is there some other way to add additional URLs to be blocked?
Errors in log I'm running the latest version of the DNS cache poisoning script with optimize=N and using all four host sources. Here are the relevant error lines: Code: Jun 14 21:24:55 MyRouter daemon.crit dnsmasq[7110]: error at line 41217 of /tmp/gen Jun 14 21:24:55 MyRouter daemon.crit dnsmasq[7110]: FAILED to start up Jun 14 21:25:13 MyRouter user.notice root: ADBLOCK List Sample Format SUCCESS Jun 14 21:25:13 MyRouter user.notice root: ADBLOCK List Contains End Total of 83133 Entries Jun 14 21:25:14 MyRouter user.notice root: ADBLOCK Dnsmasq NOT Running Dnsmasq Restarting Jun 14 21:25:14 MyRouter daemon.info dnsmasq[7138]: started, version 2.46 cachesize 4096 /tmp/gen is 0 bytes ?! Any thoughts?
I'm running Victek's 1.23.8515 ND RAF firmware, and I cannot, for the life of me, get whitelisting to work. I've tried enabling and disabling the "OPTIMISE=" line in the script, without any discernible difference. USEWHITELIST="Y" # Enter sites in format below WHITE="digg.com" Any ideas?
In looking over your code; I don't know if you are trying to optimize it anymore, but you could use arrays instead of individual variables and cut down your code by about 75% since you use separate functions for each source i.e. you could do: Code: S[0]="http://www.mvps.org/winhelp2002/hosts.txt" # ~612K S[1]="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" # ~72K S[2]="http://someonewhocares.org/hosts/hosts" # ~208K S[3]="http://hostsfile.mine.nu/Hosts" # ~2.59M GETS="Y Y N N" USEWHITELIST="N" WHITE="whitesite1.com whitesite2.com whitesite3.com whitesite4.com whitesite5.com" And for your function Code: DS () { ##(Inline grep for 127.0.0.1 & Delete DOS Format Returns) for (( i = 0 ; i < ${#S[@]} ; i++ )) do D=0 if [[ $GETS[$i] == "Y" ]] ; then if wget $S[$i] -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved S$i $S[$i] D=1 else logger ADBLOCK S$i ERROR fi fi CLN done } This will iterate through each source and run your cleanup function. Also makes it easier to add new sources at a later time I don't code in bash, so the argument syntax is borrowed from google and has not been tested; there may be some errors in it.
I'm trying the DNSmasq version after a long successful time with the hosts version and notice that if for any reason DNSmasq stops and starts, the add blocking is not relaunched. This includes a scenario where I have simply added an additional static DHCP record. Is there a way to ensure that it is running all of the time? Otherwise the script works well. :thumbup:
Hmmm... I got OPTIMISE enabled and using Victek's 1.23 build. digg.com comes up no problem (Not even in my whitelist). Have you tried rebooting your router after saving/editing the script?
By "...digg.com comes up no problem," are you saying digg.com, the website, loads up in your web browser? Because that's not my problem, my problem is that it is not whitelisting the site, as in, not blocking the ads. There are a few blogs and forums I visit that don't particularly work well when you block their ads/page elements, so I'd like to whitelist them and was trying to use digg.com to test the whitelisting to no avail.
Yes, that is correct. I added "digg.com" to the whitelist section, so as to test out how easy it was to NOT block ads on a site, using this script. End result: it still continued to block ads on digg.com. Hence my quandary. EDIT: I just realized the source of the confusion...in my earlier posts I said that whe I put it in the whitelist it was not blocking the ads, I mean to say, when I put it in the whitelist it was not unblocking ads. That is to say, whitelisting appeared to be broken. No, I did not reboot the router after making that change, I merely released/renewed the WAN IP to get DNSMasq and the script to restart.
The ads on digg.com are not from the digg.com domain which is why they aren't being let through. Check the domain of the ads (I see ads from doubleclick.net, advertising.com, and another from atdmt.com) and whitelist those domains, not digg.com.
Good point However, the browser-based ad-blocking scripts make this much easier: you whitelist a site or page, and it lets through all the ads on that page. It must dynamically check all the places from which the ads are originating and temporarily allow those through for the current page visit.
I was just wondering if anyone else had problems getting to "www.intel.com"? I had to add it into the Whitelist, but it just seems strange that intel would be in the block list at any site that provides host blocking sites for download. Thanks I'm running Tomato Firmware v1.25.0103 on a Linksys WRT54G V4
I seem to be having trouble with this script using Thor's variant of Tomato (ads are not blocked and I can still ping the domains on the blacklist). I was using Teddy Bear's variant earlier and had no problem. I see no obvious errors in the log. Please find log below - Thanks. Sep 4 06:56:28 2FAST4U user.notice root: ADBLOCK Freeing Memory Sep 4 06:56:33 2FAST4U user.notice root: ADBLOCK Retrieved S1 http://www.mvps.org/winhelp2002/hosts.txt Sep 4 06:56:46 2FAST4U user.notice root: ADBLOCK List Cleaned Sep 4 06:56:47 2FAST4U cron.err crond[117]: time disparity of 20867336 minutes detected Sep 4 06:56:48 2FAST4U user.notice root: ADBLOCK Retrieved S2 http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts Sep 4 06:56:58 2FAST4U user.notice root: ADBLOCK List Cleaned Sep 4 06:57:08 2FAST4U user.notice root: ADBLOCK Whitelist Applied Sep 4 06:57:09 2FAST4U daemon.info dnsmasq[451]: started, version 2.49 cachesize 2048 Sep 4 06:57:09 2FAST4U daemon.info dnsmasq[451]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N DHCP no-TFTP Sep 4 06:57:09 2FAST4U daemon.info dnsmasq[451]: asynchronous logging enabled, queue limit is 5 messages Sep 4 06:57:09 2FAST4U daemon.info dnsmasq-dhcp[451]: DHCP, IP range 192.168.1.100 -- 192.168.1.149, lease time 1d Sep 4 06:57:09 2FAST4U daemon.info dnsmasq[451]: reading /etc/resolv.dnsmasq Sep 4 06:57:09 2FAST4U daemon.info dnsmasq[451]: using nameserver 172.27.35.1#53 Sep 4 06:57:09 2FAST4U daemon.info dnsmasq[451]: read /etc/hosts - 2 addresses Sep 4 06:57:09 2FAST4U daemon.info dnsmasq[451]: read /etc/hosts.dnsmasq - 1 addresses Sep 4 06:57:25 2FAST4U user.notice root: ADBLOCK List Sample Format SUCCESS Sep 4 06:57:25 2FAST4U user.notice root: ADBLOCK List Contains End Total of 15960 Entries Sep 4 06:57:25 2FAST4U user.notice root: ADBLOCK Dnsmasq is Running Failsafe Ignored Sep 4 06:59:12 2FAST4U daemon.info dnsmasq[451]: exiting on receipt of SIGTERM Sep 4 06:59:12 2FAST4U daemon.info dnsmasq[503]: started, version 2.49 cachesize 150 Sep 4 06:59:12 2FAST4U daemon.info dnsmasq[503]: compile time options: no-IPv6 GNU-getopt no-RTC no-DBus no-I18N DHCP no-TFTP Sep 4 06:59:12 2FAST4U daemon.info dnsmasq-dhcp[503]: DHCP, IP range 192.168.1.100 -- 192.168.1.149, lease time 1d Sep 4 06:59:12 2FAST4U daemon.info dnsmasq[503]: reading /etc/resolv.dnsmasq Sep 4 06:59:12 2FAST4U daemon.info dnsmasq[503]: using nameserver 172.27.35.1#53 Sep 4 06:59:12 2FAST4U daemon.info dnsmasq[503]: read /etc/hosts - 2 addresses Sep 4 06:59:12 2FAST4U daemon.info dnsmasq[503]: read /etc/hosts.dnsmasq - 1 addresses
even Teddy Bear's variant does not work for me. I am using a WL-500Gv2 router. This script was running fine on my 520GU router. Anyone have this script working on a WL-500Gv2 router?
Based on ng12345's idea, I updated the script to use a loop (and I tested it ) here is the code: Code: ## ALL-U-NEED AdBlocking By YAQUI 5/1/09 ## Dnsmasq Ed. v2.92 sleep 20 ## Auto Update? (Y or N) AUPD="N" ## Create ADBLOCK.sh rm -f /tmp/ADBLOCK.sh ADB="/tmp/ADBLOCK.sh" touch $ADB ( cat <<'ENDF' #!/bin/sh ## EDITABLE VARIABLES OPTIMISE="N" GETS="1 2" S1="http://www.mvps.org/winhelp2002/hosts.txt" # ~612K S2="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" # ~72K S3="http://someonewhocares.org/hosts/hosts" # ~208K S4="http://hostsfile.mine.nu/Hosts" # ~2.59M USEWHITELIST="Y" # Enter sites in format below WHITE="editme.com editme.com editme.com" ## DO NOT EDIT BELOW NIP="0.0.0.0" GEN="/tmp/gen" TMP="/tmp/temp" ## FUNCTIONS CLR () { rm -f $GEN rm -f $TMP touch $GEN touch $TMP } FMEM () { service dnsmasq stop killall -9 dnsmasq logger ADBLOCK Freeing Memory } DS () { ##(Inline grep for 127.0.0.1 & Delete DOS Format Returns) for i in $GETS; do eval url="\$S$i" if wget $url -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved $url CLN else logger ADBLOCK ERROR FOR $url fi done } CLN () { sed -i -e 's/[[:cntrl:][:blank:]]//g' $GEN sed -i -e '/\#.*$/ s/\#.*$//' $GEN sed -i -e '/\[.*\]/ s/\[.*\]//' $GEN sed -i -e '/^$/d' $GEN sed -i -e '/127\.0\.0\.1/ s/127\.0\.0\.1//' $GEN sed -i -e '/^www[0-9]*\./ s/^www[0-9]*\.//' $GEN sed -i -e '/^[0-9]*www[0-9]*\./ s/^[0-9]*www[0-9]*\.//' $GEN sed -i -e '/^www\./ s/^www\.//' $GEN sed -i -e '/</d' $GEN sed -i -e 's/^[ \t]*//;s/[ \t]*$//' $GEN cat $GEN | sort -u > $TMP mv $TMP $GEN rm -f $TMP logger ADBLOCK List Cleaned } FDNSM () { sed -i -e 's|$|/'$NIP'|' $GEN sed -i -e 's|^|address=/|' $GEN } LCFG () { cat /etc/dnsmasq.conf >> $GEN } OPT () { if [[ $OPTIMISE == "Y" ]] ; then cat >> $GEN <<EOF cache-size=2048 log-async=5 EOF fi } LWHT () { if [[ $USEWHITELIST == "Y" ]] ; then for site in $WHITE do sed -i -e "/$site/d" $GEN done logger ADBLOCK Whitelist Applied fi } LBLK () { dnsmasq --conf-file=$GEN } TST () { sleep 15 if sed -n -e '/^address=\/ad\..*\..*\/0\.0\.0\.0$/p' $GEN ; then TOT=`wc -l $GEN | cut -d" " -f5` logger ADBLOCK List Sample Format SUCCESS logger ADBLOCK List Contains End Total of $TOT Entries else logger ADBLOCK List ERROR fi } FS () { if ps | grep -E "dnsmasq" | grep -E "nobody" ; then logger ADBLOCK Dnsmasq is Running Failsafe Ignored else logger ADBLOCK Dnsmasq NOT Running Dnsmasq Restarting service dnsmasq stop killall -9 dnsmasq dnsmasq fi } ## Run Functions CLR FMEM DS FDNSM LCFG OPT LWHT LBLK TST FS CLR ## End of ADBLOCK.sh ENDF ) > $ADB AUP () { if [[ $AUPD == "Y" ]] ; then if [[ $(cru l | grep AdUpd | cut -d "#" -f2) != "AdUpd" ]] ; then ## cru (a)dd <name> "min hr day mo wkday <cmd>" ## min=0-59 hour=0-23 day=1-31 month=1-12 sun=0 *=all cru a AdUpd "0 0 * * 2 $ADB" fi fi } ## Run ADBLOCK.sh & AUP chmod 777 $ADB $ADB AUP What has changed: * replaced DS1 to DS4 by a single function, DS, which contains the loop. * replaced GETS1 to GETS4 by an array, GETS. You can specify in GETS the index of which url you want to download, it removes a "if". For example "1 2" will download S1 and S2. * DS call CLN if wget succeeded, no need for the variable D anymore. * 3054 bytes...
instead of repeating the same code by copy/paste, the loop will repeat it for you by code. for example: say "hello 1" say "hello 2" say "hello 3" say "hello 4" with a loop: for i = 1 to 4: say "hello i" it reduces the code size, and if you want to update the code inside the loop, you won't have to update it multiple times, just once. In the previous script, if you wanted to add an url, you had to copy/paste some codes around to make it work. now you just have to add a new Sx and add a value in GETS to use it.
Any possibilities that, the blocked URL/link link to any picture from the Web GUI of the tomato? Sometimes it looks bad to see "no response from 0.0.0.0"...
you can try to redirect to another address, maybe a local address to a computer where you installed a web server (xampp) with an empty index.html. 0.0.0.0 is useful because it's faster for the OS to resolve it.
Aah.. .we're almost there... The problem is I want to keep the number of server low. At best, is it possible I only run the tomato router itself, so that the ads will be redirected to 192.168.1.1/spin.gif. Can I redirect to the tomato itself and point to a certain file (spin.gif)? Or, should I redirect, let say to 192.168.1.253 (which is actually not exists in network) but next the tomato will route 192.168.1.253 to spin.gif. (add another IP ROUTE command?)... Anybody comes up with any solution/choices?... Let say I'm a geek and willing to run any daemon,command,script,etc (with only tomato server/router to be redirected for the ads)
I don't think you can point to an image directly. dns resolves domain names only (it replaces the name with an ip). if the dns can't resolve the name, it stops there.
I don't know, there's a few others around here, but they do mostly the same thing, they won't solve your problem.
I used the code provided by srouquette and changed a few sections of the code to make it work the way I wanted it to... - Auto Update is removed, I reboot my unit once a week (this works for me) - enabled six sites to pull data from -- changed url for pdl.yoyo.org - added intel.com, webex.com and yahoo.com to the white list (change these as you need) - removed extra characters - ADBLOCK.sh no longer created - single list clean - additional logging - changed function flow - clear does not create temp file - dnsmaq optimization lines are removed -- place them in advanced -> dhcp/dns -> dnsmasq custom config ---cache-size=2048 ---log-async=5 thanks goes to YAQUI, ng12345, and srouquette; for providing the base code and the optimizations Code: ## ALL-U-NEED AdBlocking By YAQUI 5/1/09 ## Dnsmasq Ed. v2.92 ## ## Code optimizations Oct 9 2009 ## idea by ng12345 / implemented by srouquette ## ## Some bit overides taken from xcooling July 8 2008 2.1 build ## ## slightly modified Nov 8 2009 - Groosh ## pdl.yoyo.org url chng, added a few other lists, creation ADBLOCK.sh removed, single cln, added more logging, chng function flow sleep 20 #!/bin/sh ## EDITABLE VARIABLES GETS="2 3" S1="http://www.mvps.org/winhelp2002/hosts.txt" #615794B S2="http://pgl.yoyo.org/as/serverlist.php?showintro=0;hostformat=hosts" #70581B S3="http://someonewhocares.org/hosts/hosts" #215569B S4="http://hostsfile.mine.nu/Hosts" #2796873B S5="http://support.it-mate.co.uk/downloads/hosts.txt" #3429690B S6="http://hosts-file.net/hphosts-partial.asp" #75142B # Enter sites in format below WHITE="intel.com webex.com yahoo.com" ## DO NOT EDIT BELOW NIP="0.0.0.0" GEN="/tmp/gen" TMP="/tmp/temp" ## FUNCTIONS CLR () { rm -f $GEN touch $GEN logger ADBLOCK Clearing TEMP File } FMEM () { service dnsmasq stop killall -9 dnsmasq logger ADBLOCK Unloading Dnsmaq From Memory } DS () { ##(Inline grep for 127.0.0.1 & Delete DOS Format Returns) for i in $GETS; do eval url="\$S$i" if wget $url -O - | grep 127.0.0.1 | tr -d "\r" >> $GEN ; then logger ADBLOCK Retrieved $url else logger ADBLOCK ERROR Retriving $url fi done } CLN () { sed -i -e 's/[[:cntrl:][:blank:]]//g' $GEN sed -i -e '/\#.*$/ s/\#.*$//' $GEN sed -i -e '/\[.*\]/ s/\[.*\]//' $GEN sed -i -e '/^$/d' $GEN sed -i -e '/127\.0\.0\.1/ s/127\.0\.0\.1//' $GEN sed -i -e '/^www[0-9]*\./ s/^www[0-9]*\.//' $GEN sed -i -e '/^[0-9]*www[0-9]*\./ s/^[0-9]*www[0-9]*\.//' $GEN sed -i -e '/^www\./ s/^www\.//' $GEN sed -i -e '/</d' $GEN sed -i -e 's/^[ \t]*//;s/[ \t]*$//' $GEN cat $GEN | sort -u > $TMP mv $TMP $GEN rm -f $TMP logger ADBLOCK Blacklist Scrubbed } FDNSM () { sed -i -e 's|$|/'$NIP'|' $GEN sed -i -e 's|^|address=/|' $GEN logger ADBLOCK Blacklist Alignment } LCFG () { cat /etc/dnsmasq.conf >> $GEN logger ADBLOCK Dnsmasq Config Applied } LWHT () { for site in $WHITE do sed -i -e "/$site/d" $GEN done logger ADBLOCK Whitelist Applied } LBLK () { dnsmasq --conf-file=$GEN logger ADBLOCK Launching Dnsmasq With Blacklist sleep15 FS } FS () { if ps | grep -E "dnsmasq" | grep -E "nobody" ; then logger ADBLOCK Dnsmasq with Blacklist RUNNING else logger ADBLOCK Dnsmasq NOT Running starting Dnsmasq without Blacklist service dnsmasq stop killall -9 dnsmasq dnsmasq fi } TST () { if sed -n -e '/^address=\/ad\..*\..*\/0\.0\.0\.0$/p' $GEN ; then TOT=$(expr $(awk 'END { print NR }' $GEN) - 15) logger ADBLOCK Blacklist Format PASS logger ADBLOCK Blacklist Contains $TOT Entries LBLK else logger ADBLOCK Blacklist Format ERROR FS fi } ## Run Functions CLR FMEM DS CLN FDNSM LCFG LWHT TST ## Launches LBLK and/OR FS CLR
you should warn that you cron the wanup script, because some people may have other stuff in this script, like a vpn client connecting to a server, or something else.
AdUpd "30 3 * * 2 It's supposed to run every TUE on 3:30 AM? Doing what exactly? Download fresh set of host file ... It has no effect whatsoever for us having 24hr forced disconnect (ISP policy) since it reruns - autoupdates every 24h... right? or am i reading this wrong?
Yes, if you have other items firing from the wanup script your best course of action would be to surround that launch in an if statement so that it is not fired everytime the hosts files are refreshed... perhaps only when the process is not running. The other alternative would be to put back in the code which encapsulates the hosts file update and fire that independant script from the cron.
Yes it is setup to fire on tuesdays at 3:30am, which is my quiet point, there is adaquate directions on how to change the time interval if you would like a different time. The update will go out and update/refresh the blacklist when the cron triggers. If you have a connection that is reset/renewed within a one week period I would advise pulling out all of the autoupdate code altogether. You will be checking for updates regularly enough.
So i can safely just cut out autoupdate portion and AUP call on the end of the script? BTW - thx for quick answer, appreciated Cheers!
Does the Hosts Edition of this script still work properly, since it hasn't been updated in a while? Or should I just use the recently updated Dnsmasq Edition?
I have problem with autoupdate - it does nothing, nothing in logs just Code: unknown user.notice root: ADBLOCK AutoUpdate added to Cron at wan up and that's all. http://www.linksysinfo.org/forums/showthread.php?t=57556 With this script it works. No idea what's wrong with Yaqui version.
A couple of questions. What does auto-update Y or N do for the dnsmasq version? Also if I have my main router using the script and another router with a cable plugged in and then sending WDS to another router, do I need to install the script on all 3 routers, just the main router and the plugged in main WDS router, or just the main router?
yaqui, thanks for this script there is mistake in your script in autoupdate function, you wrote: Code: if [[ $(cru l | grep AdUpd | cut -d "#" -f2) != "AdUpd" ]] ; then but there must be (two quote symbols forgotten): Code: if [[ "$(cru l | grep AdUpd | cut -d "#" -f2)" != "AdUpd" ]] ; then after this edit script will be able to add cron task and autoupdate should work. travanx, autoupdate runs once a week and updates set of AD-hosts. If your main router runs as DNS server for all other clients (pc's or routers) then you need to install this script only on main router.
I don't think that mix of quotes will work, maybe using strong quotes for the inner ones would work Code: if [[ "$(cru l | grep AdUpd | cut -d '#' -f2)" != "AdUpd" ]] ; then
