HTTP SPAM Blocking - How To

Discussion in 'Tomato Firmware' started by mikester, Apr 18, 2007.


Was this useful?

Poll closed Apr 22, 2007.
  1. Yes

    8 vote(s)
  2. No

    3 vote(s)
  3. Maybe

    3 vote(s)
  1. mikester

    mikester Network Guru Member

    Do you hate wasting bandwidth and time downloading pop-ups, pop-unders, video ads, and irritating mouseovers? I do and have a pretty reliable way for Tomato to drop all HTTP requests to the spam ads.

    The Benefits:
    * the following setup kills about 99% of the unwanted spam on my network
    * web surfing is now a LOT faster because it blocks all outgoing requests for spam as opposed to blocking incomming data that has already been transfered to your computer
    * no need for you to sell your surfing habits to 3rd party proxies
    * no need to install Squid/Proxy software on your Tomato
    * blocks Torrents! If you want to allow torrents then delete the first two lines in List #1 containing "announce" and "tracker"

    It's a work in progress. If there is interest I can periodically post my updates. Please forward any suggestions and inclusions.


    HOW TO
    * Under http://[Your Tomato IP]/restrict.asp
    * Create a new rule - I use names "Keyword Blocking X"
    * Use the settings in the attached thumbnail
    * Using the "All Accept" rule will allow you to bypass the filter by MAC address
    * The lists have a maximum length - right now I have two lists/rules

    Rule List #1
    fuc* <----the "F" word

    List 2 - last updated 2008-08-11

    A good list of black listed websites can be retrieved from -> serach under host file blocking. Here is a link (700kb text file)

    An easier solution is to use an opendns server

    Attached Files:

  2. roadkill

    roadkill Super Moderator Staff Member Member

    I think it would be a cool feature to add..
    like AdBlock in Firefox
  3. Hypernova

    Hypernova LI Guru Member

    You have a few repeats in that list especially doubleclick and a few others. I think follow the examples giving in the F/W and then use the list provided in AdBlock is a better solution.
  4. StevenG

    StevenG LI Guru Member

    I gave this a shot tonight, and couldn't get it to work. I tried blocking just doubleclick as a keyword, but it still loaded it. Hmmm...

    One thing I liked with the actiontec router that came with my Fios install was that you could set up include lists, not just exclude. When it came to locking down my kids PC, it was much easier to say which sites I wanted to let them have access to vs. not have access to.
  5. GeeTek

    GeeTek Guest

    You have obviously never tried to block double click. It is impossible to have too many variations and entries to block them. They always seem to find a way around the filters. Now that google owns them you need to triple the double click entries in every single malware filter that you can.
  6. Hypernova

    Hypernova LI Guru Member

    doubleclick. <-All I used to block them and it works.
  7. GeeTek

    GeeTek Guest

    Thanks so much for the clarification. I'll drop all of my duplictes immediately.
  8. mikester

    mikester Network Guru Member

    Duplicates are easier to find especially when they are printed on a full screen editor!

    About the doubleclick ads comming through, I've found that a lot of ads come from javascripts through includes. You really need to check out the web page source code to find the bugger loading the stuff up.

    Watch out that some spam sites use keywords used in normal web programming.
    i.e. if you block the word "banner"
    would get the web page blocked even though no web content was being loaded.

    Amazon and Walmart are big offenders for spam ads but if you like to look for price comparisons online then a LOT of web pages suddenly disappear! ;-)

    Post the offending web page links and I'll have a look. My list is by no means all encompassing, just blocks the websites I hit.

    I cleaned up and sorted my list in the original post and saved the changes. Now fits on one list! Keep in mind there's a maximum number of characters allowed in the list.
  9. Int15

    Int15 Network Guru Member

    Good job, thanks!
    However, why are you blocking

    Thanks again,
  10. mikester

    mikester Network Guru Member

    I use the COB rule ;-)

    Basically block sites/content if they have "popped up" somehow/someway that is irrelevant to my "internet experience".

    I find aspalliance serves/posts too much ad&spam content on other scripting/programming websites.

    Change it to "" and it will block some of the spam content but still show you the basic web site.
  11. Int15

    Int15 Network Guru Member

    Great, thanks again!

  12. lwf-

    lwf- Network Guru Member

  13. yaqui

    yaqui LI Guru Member

  14. mikester

    mikester Network Guru Member

    All good links but I think I prefer the "small is beautiful" philosophy of Tomato. Personally I don't want to install and maintain spam buster software on every computer on the network...just call me lazy
  15. larsrya8

    larsrya8 LI Guru Member

    Adblock Plus for Firefox maintains itself.
  16. yaqui

    yaqui LI Guru Member

  17. paped

    paped LI Guru Member

    The list seems to work great.... thanks very much....
  18. yaqui

    yaqui LI Guru Member

    How much load

    Has anyone studied how much load using lists like this puts on the router?

    Wondering if it is better to do it on a 'per machine' basis.
  19. mikester

    mikester Network Guru Member

    Here's some stats for you:

    WRT54GL running Tomato v.1.05.0977
    CPU Load (1 / 5 / 15 mins) 3.91 / 4.49 / 4.29 (154 connections, mostly web surfing)
    Total / Free Memory 14.20 MB / 1,432.00 KB (9.85%)
    Uptime 16 days, 22:38:50
  20. mikester

    mikester Network Guru Member

    Just for fun I chose a website I like to view but HATE the spam content.

    I ran a test using IE6. Before each test I went to "Tools" - "Internet Options", "deleted cookies", "delete files + all offline content", "clear history". I didn't bother timing the differences as I think the results are self explainatory.


    Data Transfer Stats:
    Tomato Spam Filter OFF
    UL: 84014 bytes
    DL: 777609 bytes

    Tomato Spam Filter ON
    UL: 49281 bytes
    DL: 442139 bytes

    Thats roughly a 44% reduction in data transfered. It's like getting an extra 44% of bandwidth!

    Hey Yaqui, how about you show some comparison stats along with some time comparisons?

    Flame away!
  21. larsrya8

    larsrya8 LI Guru Member

    Those load numbers are really high... I thought it wasn't supposed to go over 1.00?
  22. yaqui

    yaqui LI Guru Member

    Was this just using one machine and having 154 simultaneous connections going?

    I would like to know what happens with you have multiple client machines (like more than 10) doing many http requests.

    I think the load distributed with each machine doing the filtering would be better, rather than the router doing all the work for all 10 machines.

    Maybe this is wrong thinking, because the router sees all the requests the same as it would with just one client machine?
  23. mikester

    mikester Network Guru Member

    1 wireless and 2 wired machines were connected at the time.

    I don't think I've ever seen load numbers below 1.00. Can you post yours for comparison?
  24. yaqui

    yaqui LI Guru Member

    How do I isolate just measuring the filtering process? The load is taking into account all the routing functions too... isn't it??

    I'm seeing higher numbers with each additional machine I connect.
    I don't know if there is anyway to just isolate the measurement like I said.... maybe there is some way with the 'top' command?
  25. larsrya8

    larsrya8 LI Guru Member

    Uptime 11 days, 21:51:08
    CPU Load (1 / 5 / 15 mins) 0.00 / 0.01 / 0.00
    Total / Free Memory 14.20/3,320.00 (22.83%)

    This is with Conntrack reporting ~370 connections. Six computers connected (5 wired, 1 wireless). Bittorrent running on at least one.

    I have QOS, but I'm only using ports to classify.. no L7. Also using WPA on the wireless.
  26. mikester

    mikester Network Guru Member cpu load is pretty consistent with or without the keyword blocking running. Maybe because I'm also running a WDS link, QOS, cifs file shares, remote syslog and SNMP?

    My other router load is
    CPU Load (1 / 5 / 15 mins) 0.12 / 0.03 / 0.02
  27. tunasashimi

    tunasashimi LI Guru Member

    If you're not seeing lag with those load averages, then..... you must be using dialup :D

    Does anybody have some latency measurements, or does tomato have some really cool hardcoded nice values for processes?
  28. snwbdr

    snwbdr Network Guru Member

    You had up there. I use it because it is free with comcast. I'm cheap. Anyway it blocks updates to the antivirus and firewall. I should have read the list first, but how is mcafee spam? Anyway I just deleted it from the list everything else works good, but not sure what everything on there is. Anyone try adblock plus's list? Does it work well or not? thanks
  29. yaqui

    yaqui LI Guru Member

  30. dangdonkey

    dangdonkey Network Guru Member

    To your network yes but it still makes it to the router where it's dropped.
  31. larsrya8

    larsrya8 LI Guru Member

    Are you sure it doesn't block the outgoing connections to those sites? You may be confusing this with incoming QOS.
  32. jochen

    jochen Network Guru Member

    I'm looking for a similar solution for parental control for the kids. Any ideas how to do this? I think it is no so easy to do this based on URLs.
    My understanding of tomatos http blocking is, that it is "blacklist" based (all URLs matching the keywords are blocked). I would prefer a "whitelist" based solution.
  33. yaqui

    yaqui LI Guru Member

    You can try FoxFilter

    It will filter keywords in the url.

    Edit: Sorry, I see that you want a whitelist. Maybe there can be some way to use wildcards in access restriction to block all sites... then set up another access restriction with a listing of only they sites you want to allow. (something like: ^*.*.*$ or simply ^$ for all sites?)

    Or, there may be a way in FoxFilter to use wildcards since that will allow URL Exceptions, I'm not sure yet if\how you can wildcard in that.

    I don't see "allow URL exceptions" yet in Tomato. Maybe the author(s) can add these features?
  34. Hi,
    I am using your HTTP SPAM Blocking Rule List #1, which is great. Thank you.
    You say in your post that right now you use two lists/rules. Could you please send or post the other one for me.
  35. wycf

    wycf Network Guru Member

  36. mikester

    mikester Network Guru Member

    try using


    in your list

    Right now only one list is needed - I eliminated duplicates
  37. Talon88

    Talon88 LI Guru Member

  38. wycf

    wycf Network Guru Member

    You are kidding, right?
  39. wycf

    wycf Network Guru Member

    ok, that works, partially.

    how can I block those gif images under
    they are annoying.
  40. Talon88

    Talon88 LI Guru Member


    Oh, I made a mistake. I miss read you want to
    block that site....


  41. wycf

    wycf Network Guru Member

    The FAQ says:
    "Some limitations: Hostname is a separate string from path?query (path and query are considered as one string), so you can't use "". Others, like the POST data, or the content of the requested pages are not checked. Escaped characters are not decoded."

    I am not understand this very clear.

    So if I want block juat part of a website, for example, all the pages under, what should I do?
  42. yaqui

    yaqui LI Guru Member

    If you are blocking the advertising company's domain name (ie. doubleclick)... then all ads/web spam associated with that domain will be blocked regardless of any sub-path/url you may be visiting.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice