1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

ALL-U-NEED Ad Blocking

Discussion in 'Tomato Firmware' started by srouquette, Nov 6, 2010.

  1. ppsun Reformed Router

    Still works. But the host list may need some updating/fine-tuning to get optimal blocking (ie: not to overblock or underblock sites that you visit). What's your hardware and firmware configuration?
  2. Klakinoumi Reformed Router

    First of all, thanks for the awesome work you're all doing guys. I'am amazed of how great this thing is.

    However, i encounter a problem with my wnr3500L v2 running on Tomato Firmware 1.28.0000 MIPSR2-101 K26 USB AIO and i'm not seeing where i'm doing something wrong.

    I'am trying to add this list : https://easylist-downloads.adblockplus.org/liste_fr easylist.txt to my adblock script.

    Here is my config :

    Code:
    OPTIMISE="Y"
    GETS="1 2 3"
    TRIM_BEGIN=4
    S1="http://pgl.yoyo.org/as/serverlist.php?hostformat=nohtml"  #44K
    S2="http://mirror1.malwaredomains.com/files/justdomains"    #189K
    S3="https://easylist-downloads.adblockplus.org/liste_fr+easylist.txt" #600K
    S4="http://www.malwaredomainlist.com/hostslist/hosts.txt"  #97K
    S5="http://winhelp2002.mvps.org/hosts.txt"              #620K
    S6="http://hosts-file.net/hphosts-partial.asp"              #460K
    S7="http://hostsfile.mine.nu/Hosts"                        #2641K
    S8="http://support.it-mate.co.uk/downloads/hosts.txt"      #3851K
    And here is the log for this config :
    Code:
    Oct 17 00:20:26 MYRouter daemon.info dnsmasq-dhcp[664]: DHCPREQUEST(br0) 192.168.1.92 xxxxxxxxxxxxxx
    Oct 17 00:20:26 MYRouter daemon.info dnsmasq-dhcp[664]: DHCPACK(br0) 192.168.1.92 xxxxxxxxxxxxxx iPadinoumi
    Oct 17 00:20:37 MYRouter daemon.info pixelserv[1039]: /tmp/mnt/CYGOGNE/pixelserv V18 compiled: Nov  3 2010 21:03:21 from pixelserv18.c
    Oct 17 00:20:37 MYRouter daemon.notice pixelserv[1041]: Listening on 192.168.1.7:80
    Oct 17 00:20:40 MYRouter user.notice root: ADBLOCK: http://pgl.yoyo.org/as/serverlist.php?hostformat=nohtml
    Oct 17 00:20:41 MYRouter user.notice root: ADBLOCK: http://mirror1.malwaredomains.com/files/justdomains
    Oct 17 00:20:42 MYRouter user.notice root: ADBLOCK: https://easylist-downloads.adblockplus.org/liste_fr+easylist.txt
    Oct 17 00:20:42 MYRouter user.notice root: ADBLOCK: 12650 entries
    Oct 17 00:20:42 MYRouter user.notice root: ADBLOCK: sorting hosts...
    Oct 17 00:20:56 MYRouter daemon.info dnsmasq-dhcp[664]: DHCPREQUEST(br0) 192.168.1.75 5xxxxxxxxxxxxxx
    Oct 17 00:20:56 MYRouter daemon.info dnsmasq-dhcp[664]: DHCPACK(br0) 192.168.1.75 xxxxxxxxxxxxxx iPhonoumi
    Oct 17 00:21:05 MYRouter user.notice root: ADBLOCK: hosts sorted.
    Oct 17 00:21:05 MYRouter user.notice root: ADBLOCK: 12534 entries
    Oct 17 00:21:05 MYRouter cron.err crond[632]: time disparity of 22507100 minutes detected
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[664]: exiting on receipt of SIGTERM
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[1154]: started, version 2.61 cachesize 150
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[1154]: compile time options: IPv6 GNU-getopt no-RTC no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq-dhcp[1154]: DHCP, IP range 192.168.1.50 -- 192.168.1.100, lease time 1d
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[1154]: reading /etc/resolv.dnsmasq
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[1154]: using nameserver 212.27.40.241#53
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[1154]: using nameserver 212.27.40.240#53
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[1154]: read /etc/hosts - 2 addresses
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq[1154]: read /etc/dnsmasq/hosts/hosts - 5 addresses
    Oct 17 00:21:05 MYRouter daemon.info dnsmasq-dhcp[1154]: read /etc/dnsmasq/dhcp/dhcp-hosts
    Oct 17 00:21:05 MYRouter user.notice root: ADBLOCK: dnsmasq is running
    
    12534 entries

    But it clearly doesn't take the easylist in count because with the exact same file located on my own server, this is the log I was given :

    Code:
    Oct 17 00:09:15 MYRouter daemon.notice pixelserv[1061]: Listening on 192.168.1.7:80
    Oct 17 00:09:17 MYRouter user.notice root: ADBLOCK: http://pgl.yoyo.org/as/serverlist.php?hostformat=nohtml
    Oct 17 00:09:19 MYRouter user.notice root: ADBLOCK: http://mirror1.malwaredomains.com/files/justdomains
    Oct 17 00:09:21 MYRouter user.notice root: ADBLOCK: http://klakinoumi.com/Uploads/liste_fr+easylist.txt
    Oct 17 00:09:21 MYRouter user.notice root: ADBLOCK: 32698 entries
    Oct 17 00:09:21 MYRouter user.notice root: ADBLOCK: sorting hosts...
    Oct 17 00:09:35 MYRouter daemon.info dnsmasq-dhcp[660]: DHCPREQUEST(br0) 192.168.1.75 xxxxxxxxxxxxxxx
    Oct 17 00:09:35 MYRouter daemon.info dnsmasq-dhcp[660]: DHCPACK(br0) 192.168.1.75 xxxxxxxxxxxxxxxx iPhonoumi
    Oct 17 00:09:42 MYRouter cron.err crond[632]: time disparity of 22507088 minutes detected
    Oct 17 00:10:28 MYRouter user.notice root: ADBLOCK: hosts sorted.
    Oct 17 00:10:28 MYRouter user.notice root: ADBLOCK: 32573 entries
    Oct 17 00:10:29 MYRouter daemon.info dnsmasq[660]: exiting on receipt of SIGTERM
    Oct 17 00:10:29 MYRouter daemon.crit dnsmasq[1179]: error at line 15 of /tmp/gen
    Oct 17 00:10:29 MYRouter daemon.crit dnsmasq[1179]: FAILED to start up
    Oct 17 00:10:29 MYRouter user.notice root: ADBLOCK ERROR: restarting dnsmasq...
    Oct 17 00:10:29 MYRouter daemon.info dnsmasq[1186]: started, version 2.61 cachesize 150
    Oct 17 00:10:29 MYRouter daemon.info dnsmasq[1186]: compile time options: IPv6 GNU-getopt no-RTC no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack
    Oct 17 00:10:29 MYRouter daemon.info dnsmasq-dhcp[1186]: DHCP, IP range 192.168.1.50 -- 192.168.1.100, lease time 1d
    Oct 17 00:10:29 MYRouter daemon.info dnsmasq[1186]: reading /etc/resolv.dnsmasq
    Oct 17 00:10:29 MYRouter daemon.info dnsmasq[1186]: using nameserver 212.27.40.241#53
    Oct 17 00:10:30 MYRouter daemon.info dnsmasq[1186]: using nameserver 212.27.40.240#53
    Oct 17 00:10:30 MYRouter daemon.info dnsmasq[1186]: read /etc/hosts - 2 addresses
    Oct 17 00:10:30 MYRouter daemon.info dnsmasq[1186]: read /etc/dnsmasq/hosts/hosts - 5 addresses
    Oct 17 00:10:30 MYRouter daemon.info dnsmasq-dhcp[1186]: read /etc/dnsmasq/dhcp/dhcp-hosts
    
    32573 entries

    So there are two issues here :

    1. Why when the list is on https://easylist-downloads.adblockplus.org/liste_fr easylist.txt the script is not scrapping it ? (httpS ?)
    2. Why when the list is on http://klakinoumi.com/Uploads/liste_fr easylist.txt the script is scrapping it BUT fails to end the configuration of dnsmasq and relaunch it without ADBLOCK ? (the real question here is : Why it fails ?)
    Trimming for this list or not doesn't seem to change anything.

    I hope you guys can help me, and i think some people here may have a configuration with an easylist not loading at all.

    PS: I'm absolutely sorry for my English as it's not my native tongue. :(
  3. ppsun Reformed Router

  4. Klakinoumi Reformed Router

    I'm sorry, the exact link is : http://klakinoumi.com/Uploads/liste_fr easylist.txt and it is the url that I set in my router.

    Marshcroft was saying here that it was ok to add an easylist. Gijs73 too.

    As it is basically the only one working in France if I can make it to work properly it could be a HUGE advantage.
  5. Mr. Scary Networkin' Nut

    Asus RT-N66U, Tomato Firmware v1.28.0500 MIPSR2Toastman-RT-N K26 USB VPN
  6. ppsun Reformed Router

    The klakinoumi list still doesn't load. I installed the easylist list from you link on my router (RT-N66U); the entries not recognised even before the entries sorting. I guess you will have to ask the other 2 posters how they got their easylist entries loaded.

    I noticed that you only have 12k sorted entries. Perhaps my host list might work better for you (34k sorted entries):

    TRIM_BEGIN=2
    S1="http://mirror1.malwaredomains.com/files/justdomains"
    S2="http://www.malwaredomainlist.com/hostslist/hosts.txt"
    S3="http://someonewhocares.org/hosts/hosts"
    S4="http://winhelp2002.mvps.org/hosts.txt"
    S5="http://sysctl.org/cameleon/hosts"
    S6="http://hosts-file.net/ad_servers.asp"
  7. ppsun Reformed Router

    I have the same router, but using shibby's firmware instead (but that should not matter). Replaced my current script with the original script from page 1. It loaded fine (21k sorted entries) and works. The ads were rejected and replaced with non-pixelserved connection error banners.

    I think something is not quite right on your end.
  8. Mr. Scary Networkin' Nut

    I happen to be in France, so that may have something to do with it judging by Klakinoumi's post.
  9. mstombs Network Guru

    I have posted a new version of pixelserv in the thread http://www.linksysinfo.org/index.php?threads/pixelserv-compiled-to-run-on-router-wrt54g.30509/page-2, main purpose is to publish a version with optional image loaded from a file, useful for testing/development. Note that the new v30 not yet compatible with the script in this thread - because I have not compiled in the option to restrict operation to a named interface i.e. "-n br0" as found this has an issue with tomato web gui config changes - pixelserv can get stuck listening only on an obsolete interface, and I think its easier to drop the option than to find out how to fix...
  10. tido Serious Server

    I've pasted the script from the front page and pasted it under "WAN Up". Tried to test it by going to the URL http://milliondollarhomepage.com/, are the contents of this URL are not suppose to be blocked? My show up NP. Does this mean the script is not working? Do you have any URL examples where I can test the script?

    Cheers
  11. Frequenzy Serious Server

    i've used the script and it works ok. unfortunately it slows down appstore downloads from PC or ipad. i removed it and it appstore works fine.
  12. teo yabgu Reformed Router

    I am using Ad Blocking script v3.9e and pixelserv v27(both through this forum) with Tomato Firmware v1.28.9054 MIPSR2-beta K26 USB Ext on Asus rt-n16. Everything works great except one thing. When I am browsing https links, I have noticed pixelserv cannot replace the ads on chrome or firefox.( ex: www.youtube.com is working fine but https://www.youtube.com, ads are blocked but "Unable to Connect" error cannot be replaced by 1x1 image. However, I tired with safari and both http and https links are fine with pixelserv. any ideas what I missing here?
  13. teo yabgu Reformed Router

    Here is a screenshot of my problem, I believe it explains better.[IMG]
  14. mstombs Network Guru

    Pixelserv can't currently do anything with https and doesnt try - there will be nothing on the router listening on the https 443 port. The http null gif is not a valid https response. Different browsers will display the error in different ways.
  15. teo yabgu Reformed Router

    so it is working how it is suppose to be working... I have been looking for how to change to listening port but there was nothing in the options of pixelserv(tried every version of it on this forum). Thank you so much to clear it up.
  16. mstombs Network Guru

    Port selection is in the source code of pixelserv.c but it is a compile time option - and proved not to be useful.
  17. teo yabgu Reformed Router

    I have found pixelserv v28 compiled by you for test in a different forum(dd-wrt forum) and I assume it has the option(-p) to change the port but now I am just doing it to learn more stuff and experience. I will also look into the source code too.
  18. Frequenzy Serious Server

    how to restart pixelserv?
  19. mstombs Network Guru

    From the command line see what command was used to start it with "ps"
    then
    Code:
    killall pixelserv
    then give the original command which will be something like
    Code:
    /jffs/pixelserv 192.168.1.2
    It's a good idea to do also put these commands in your firewall script, since this avoids a bug in V27!
  20. tismon Reformed Router

    If I may ask, what are the most current versions that work well together and where can they be found?
  21. tismon Reformed Router

    Hmm, can't edit. Sorry for the double post then.

    After reading through a good number of threads, it seems that the AdBlock 3.9e in the OP is still up to date and pixelserv v27 is the newest that it can use (v18 is linked in the OP). To help other newbies, there is a v30, but it isn't compatible.

    Now onto the next issue. Is it possible to save AdBlock as a script in JFFS and run that in the WAN Up entry? So far, running it as a script in the system box nets me a few errors.

    /jffs/Scripts/AllYouNeed.sh: line 6: : not found
    sleep: invalid number '10 '
    /jffs/Scripts/AllYouNeed.sh: line 8: : not found
    /jffs/Scripts/AllYouNeed.sh: line 10: { : not found

    I'm still going to try a few things, but I'm not too sure what would cause this.
  22. Bird333 LI Guru

    I would like to run from a file too.
    I tried to save to /opt/etc/config as a .wanup file but it doesn't seem to run.
  23. tismon Reformed Router

    I can put the sleep command external to the script call:
    sleep 10
    /jffs/Scripts/AllYouNeed.sh

    And remove the error blanks, and even the brackets (probably needed, though maybe not in this case) and won't get any errors, but I still don't get the output that I do when simply pasting the entire script from the system cmd. I think that I'm missing something fundamental here.

    Also, would running it from the firewall script section remove the need for the sleep command or would it just cause more issues?
  24. QSxx LI Guru

    Hosts List Update

    Here's the updated list of hosts for those of you that still use this wonderful script.
    It has been verified for errors at the time of posting and combined downloads around 250 000 entries which will, after pruning, yield around 110 000 blocked hosts. (In my case, it takes moderately loaded RT-N16 around 9 minutes to do the pruning and sorting but ofcourse YMMV).

    Code:
    TRIM_BEGIN=1
    S1="http://www.abuse.ch/spyeyetracker/blocklist.php?download=domainblocklist" # ~150
    S2="http://www.abuse.ch/zeustracker/blocklist.php?download=domainblocklist" # ~800
    S3="http://www.malwaredomainlist.com/hostslist/hosts.txt" # ~1200
    S4="http://pgl.yoyo.org/as/serverlist.php?hostformat=nohtml" # ~2500
    S5="http://hosts-file.net/hphosts-partial.asp" # ~6400
    S6="http://hosts-file.net/ad_servers.asp" # ~12200
    S7="http://mirror1.malwaredomains.com/files/justdomains" # ~13900
    S8="http://winhelp2002.mvps.org/hosts.txt" # ~14000
    S9="http://sysctl.org/cameleon/hosts" # ~20000
    S10="http://hostsfile.mine.nu/Hosts" # ~94500
    S11="http://support.it-mate.co.uk/downloads/hosts.txt" # ~185000
    N.B. This little commented number represents number of downloaded hosts as reported by router

    IMPORTANT!!! Hostlist no. 10 [http://hostsfile.mine.nu/Hosts] breaks google apps and chromium sync as it has dedicated section blocking just theese services. DO NOT include it if you use google stuff...
  25. eahm Addicted to LI

    windozer likes this.
  26. Elfew Networkin' Nut

    Please can somebody give me instructions how to set this script on my asus rt16n? Step by step for noobs ;) I cannot run this script and I dont know what I am doing wrong
  27. Frequenzy Serious Server

    the steps are at the first post. do you get any errors when you follow the instructions?
  28. Frequenzy Serious Server

    not sure if it was discussed already but is there a way to restart the ADBLOCK script without restarting the router.
  29. randyoo Serious Server

    This is a WAN Up script, so you can cause it to run by bringing your WAN connection down and back up again. When you first log in to Tomato, you should be at Status, Overview. There's a button in the WAN section that you can click to disconnect, and then you should be able to hit connect immediately afterward.
  30. eahm Addicted to LI

    The one I use is just "/tmp/dlhosts" on Tools/System, this one I'm not sure.
  31. Frequenzy Serious Server

    thanks for suggestions. will try it
  32. QSxx LI Guru

    Just cycle your connection. IF your all-you-need script is in WAN Up field...
  33. eahm Addicted to LI

    and what if you can't?
  34. QSxx LI Guru

    Easy dude... just offering a suggestion...
  35. eahm Addicted to LI

    Irrelevant, of course it works if you restart the connection or the router.
  36. QSxx LI Guru

    True... That would be true assuming the user KNOWS that WAN Up script is actually ran when "WAN" interface goes "UP".

    Otherwise user can run the script typing following into Tools / System / Execute
    Code:
    /tmp/script_wanup.sh
    in case he wants whole WAN Up section or alternatively
    Code:
    sh /tmp/ADBLOCK.sh
    if he wishes to run ONLY adblock part (because some users have other stuff in their WAN Up section).

    Yes, you can navigate away from the page once script starts running - no need to wait for it to "finish".

    Moreof... for ease-of-use same commands can be used from Administration / Buttons/LED if your router supports that function.

    In the end, I think all of this belongs to Tomato Wiki rather than individual threads as it can be applied to any script from Administration section. This is actually basic shell stuff...
    eahm likes this.
  37. 68rustang Reformed Router

    I am hoping somebody here can help me figure out why this stopped working on my router, WNR3500Lv2.

    I originally set this up a few weeks ago and it worked great. Recently it first appeared that PixelServ was having issues because I was getting the "cannot display" messages where ads were supposed to be and now it doesn't appear to be blocking anything.

    I have tried updating to Shibby 106 but that didn't change anything so I went back to 105. I have tried updating the lists but that made it worse if anything. I can browse to 192.168.2.254 (pixelserv address) and see a blank page so it appears to be running. I have checked and the ads are being served by servers that are in the lists. I have restarted the router.

    The only thing I have changed between setting it up and now is configuring OpenVPN. Would that have any effect on ad blocking?

    How can I troubleshoot this to see what is causing the problem?
  38. 68rustang Reformed Router

    FYI - #11 blocks Dropbox.
  39. Frequenzy Serious Server

    Thanks. looks like sh /tmp/ADBLOCK.sh is the best option for me
  40. Frequenzy Serious Server

    i'm also using the script. pixelserv has a bug. sometimes when we change settings on tomato gui, pixelserv goes down and never recovers. but i believe it should be resolved when you restart the router. do you see any errors on the logs?
  41. 68rustang Reformed Router

    That may be what I am seeing as well. I will look at the logs tonight and see what I can see. After troubleshooting my dropbox issue last night I think it was just taking a REALLY long time to download and optimize the lists. I am only getting 1-9 now and things seem much better. I have used dd-wrt for years but just recently switched to TomatoUSB so it all feels foreign right now.
  42. mstombs Network Guru

    v28 has this issue, one workaround is to restart pixelserv in the firewall script which does get called on those gui changes. Or hopefully update the script in first post to not use the "-n br0" call parameter. It seems the interface id can change, and I don't know how to trap.
  43. 68rustang Reformed Router

    How would I do this?
  44. Frequenzy Serious Server

    i manually restart pixelserv by typing

    killall pixelserv
    /tmp/mnt/sda1/pixelserv 192.168.1.254
  45. Bird333 LI Guru

    Is it possible to stop and start this script manually? Occasionally, I need to stop the script from working.
  46. se4587 Reformed Router

    Is there a way to have my whitelist in a text file on my jffs partition? I have an E3000 with tomato 1.28. I am using pastebin for a remote whitelist but I had a problem recently when pastebin was down so none of my white listed sites got unblocked. I'm sure there's a way to edit the script to look in /jffs/whitelist.txt with my white sites but I don't know enough about scripting to get it to work.

    Ok, well I think I got it to work. I made a file with my sites to be white listed at /jffs/whitelist.txt. I changed the Remote address to the same. Then I had to change the base 64'd section under LWHT(); I changed this line
    if wget $WURL -O - | tr -d "\r" > $WFILE ; then
    to this
    if cat $WURL | tr -d "\r" > $WFILE ; then

    Restarted & forced adblock to reload all my lists and so far it seems to be working...of course I had to copy & paste the script & run it to get the base 64 version to paste back into the scripts page...
  47. rs232 Addicted to LI

    I'm trying to run the latest version of the script (3.9e) but it gives me the following error:

    Code:
    /tmp/ADBLOCK.sh: line 78: syntax error: unexpected end of file (expecting "}")
    line 78 is actually in the base64 part of the script... Any input?

    Thanks

    Edited:
    Forget about this, I re-pasted the script and now it works.
    Thanks any ways
  48. Bob Tobalina Reformed Router

    Hi,

    How do i completely disable the script or erase it?

    I removed the lines from Administration -> Scripts > Wan Up but the script still continues to execute, using latest shibby v108, the log files still show the script I thought I erased still executing.

    Edit:

    Ok, I used telnet and accessed files on router. I did a cat on the wanup script file and it showed all the text I thought I deleted. I then had to search some telnet commands and did a vi on the wanup script and deleted line by line (dont know how to select all) then saved it and seems to be working now.

Share This Page