1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Script: Clean, Lean and Mean Adblocking

Discussion in 'Tomato Firmware' started by haarp, Apr 23, 2013.

  1. jerrm

    jerrm Network Guru Member

    It's intended to be a drop in replacement. No config changes should be necessary, but read the notes.
     
    Goggy likes this.
  2. Scottmsu

    Scottmsu Reformed Router Member

    I'm very new to Tomatousb and I have a silly question.

    I need to have 2 commands in the WAN up section of the administration scripts. I'm wondering if this is correct because I'm trying to gauge if the other script (not the adblock one) is working or i'm just not formatting it correctly. Thanks
     

    Attached Files:

  3. kthaddock

    kthaddock Network Guru Member

     
  4. koitsu

    koitsu Network Guru Member

    That should be modprobe xt_DSCP not insmod xt_DSCP.ko. I won't go into the difference between the two here.

    Note to readers: the xt_DSCP "stuff" has no relevancy/relation to adblocking, so please do not let this nor the previous post confuse you.
     
  5. Scottmsu

    Scottmsu Reformed Router Member

    @kthaddock

    Shouldn't i put the insmod xt_DSCP.ko in the init part of the administration scripts rather than in the WAN up? I'm kind of confused.

    I'm currently using insmod xt_DSCP.ko in the init section of the administration scripts and it seems to be working. I ended up deleting the adblock.sh to double check.

    For the WAN up section can I enter the commands like this and have them both execute as intended?

    /jffs/adblock/adblock.sh
    iptables -t mangle -A PREROUTING -i `nvram get wan_iface` -j DSCP --set-dscp 0

    -------------------------------
    I'm currently running a WDS between two tomatousb routers running AdvancedTomato. Do I have to add the commands for each of the router administration scripts? Or can I just input it in one router and it applies to the other automatically since they are linked together?
     
  6. koitsu

    koitsu Network Guru Member

    Each router must have its Scripts section modified. There is no "transmission" of NVRAM data between routers on a WDS network.
     
  7. jerrm

    jerrm Network Guru Member

    This is way, way off topic for this thread, but the iptables rule should be in the firewall section, not wanup. Any number of events will wipe out and reset iptables rules and will not execute the wanup script.

    Yes, the module should probably be loaded only once in init, but it shouldn't hurt if placed with the iptables rule.

    Koitsu is right about there being no replication of nvram data, but the adblock and DSCP commands should only be needed on the gateway router - assuming the gateway is serving DNS and the wan is disabled on the other WDS nodes.
     
  8. kthaddock

    kthaddock Network Guru Member

    btw I use insmode and it's working just fine.
    This is what you say in other thread you gave some advice:
    http://www.linksysinfo.org/index.ph...s-on-comcast-cisco-e-series-and-others.69006/
     
    Last edited: Jan 7, 2014
  9. koitsu

    koitsu Network Guru Member

    TL;DR -- insmod is a bad habit to get into, just use modprobe.
     
  10. leandroong

    leandroong Addicted to LI Member

    In my experience, both seems working fine. My audio device driver installed using "insmod" and bcm_nat with "modprobe".
     
  11. Elfew

    Elfew Addicted to LI Member

  12. jerrm

    jerrm Network Guru Member

    It's fine, uses the hosts files method, which may not be as aggressive at blocking, but I've pretty much decided to use hosts files myself.
    Biggest issues I see:
    Runs on the router's LAN IP, so the admin interface can't be on port 80
    No handling of non port 80 traffic - ssl timeouts will be an issue
    Hardcoded URL list
    No whitelist options
     
  13. tomatosoup

    tomatosoup Serious Server Member

    You're right, jerrm ;-)
    I did this in 2 days, and it's doing it's job for me (for now).

    I'm aware that there are better solutions available, but my intent was to provide a simple solution for people who are not that good in copying/modifying scripts from here. There are still lots of things to do, like you mentioned (whitelists, ...), but it's a start. Let's see if/when I can make some improvements - or maybe someone else also can contribute to this solution...
     
    Elfew likes this.
  14. Elfew

    Elfew Addicted to LI Member

    It is basic - it needs some.more love and features from this script - whitelist, update frequency, maybe add support for turn of adblock for devices (ip mac based).

    Anyway good start, keep good work
     
  15. jerrm

    jerrm Network Guru Member

    Completely understand. Did not mean to offend in any way. Having it built-in definitely has advantages.

    My version of this thread's namesake is not so lean anymore, but there are things I've done for the sake of compatibility that I would remove. Others things I have done because folks have some unexpected implementation methods that a built-in would not have to deal with. A fresh start is not a bad thing.
     
  16. Elfew

    Elfew Addicted to LI Member

    Just merge your knowledge, cooperate and make it user friendly. I can imagine that it could be a part of every tomato build in future. Now you are at the beggining but you made a first step. Good luck!
     
  17. tomatosoup

    tomatosoup Serious Server Member

    None taken - I know what you meant, and I know there are still lots of things which can be improved.
     
  18. Anubis14

    Anubis14 Addicted to LI Member

    Hello gentlemen, I have the script posted from OP running. I'm not too knowledgeable in these but how do I update pixelserv? my current version is v31.
     
  19. ShinichiYao

    ShinichiYao Reformed Router Member

    http://pastebin.com/raw.php?i=ziiG4zcm

    Paste this into the box on Tools->System and press Execute. Remember to Change the $PREFIX first.
     
    Scottmsu likes this.
  20. Anubis14

    Anubis14 Addicted to LI Member

    Thank you Shinichi! it worked!
     
  21. Kye-U

    Kye-U Addicted to LI Member

    I wouldn't necessarily say jerrm is at the beginning; he's actually made more progress with this than I could have ever done. His work is appreciated.
     
  22. jondlou

    jondlou Networkin' Nut Member

    Please do. I am troubleshooting but it would be nice to see what you figured out. Thanks!
     
  23. Mihai Olimpiu-Cristian

    Mihai Olimpiu-Cristian Serious Server Member

    I just want to add some experience from a DD-WRT router I use (without pixelserv - I still have to do a USB mod on that router).
    If I added Google's static DNS's 8.8.8.8 and 8.8.4.4 the slowdown is back, deleting the static DNS's fixed it, and the ugly box with connection was reset appears immediately!
    In Tomato (WITH PIXEL SERV and Jerrm's script), I can use Google's static DNS's without any problem but that's why Pixel Serv is there, right?
    I don't know why, maybe DD-WRT's script works different but it works the same "at a basic level".
    Hope it helps someone!
     
  24. Elfew

    Elfew Addicted to LI Member

    Ask in ddwrt forum...
     
  25. darkknight93

    darkknight93 Networkin' Nut Member

    Due i'm really tired right now Ill do just quick steps for AC68U:

    enable jffs and downloadmaster to get Access to /opt
    see Google or AsusMerlin howto's for that, basically in Administatration enable jffs and Format. afterwards Format your usb drive like: http://uk.nyclee.net/2012/04/28/installing-optware-in-tomato-usb-shibby/ (just fdisk and second paragraph with mke2fs for partitioning)

    so I mounted a second Partition on my usb stick to /mnt/DATA
    my adblock script is in /mnt/DATA/scripts/adblock (Contents see zip file)
    copy them, EDIT CONFIG FILE! and run sh /mnt/DATA/scripts/adblock/adblock.sh via SSH e.g.

    if you want to have adblock sheduled add to /jffs/scripts/post-Mount (Access 0777):
    Code:
    #!/bin/sh
    if [ -d "/mnt/DATA/scripts" ]
    then
    cru a Adblock  "0 0 * * * sh /mnt/DATA/scripts/adblock/adblock.sh force"
    fi
    
    maybe you will Need following lines in /jffs/scripts/firewall-start (Access 0777):
    Code:
    #!/bin/sh
    iptables -t nat -A PREROUTING -p tcp -d 192.168.1.250 --dport 80 -j DNAT --to 192.168.1.250:8080
    
    But only if Services are restarted and iptables are flushed/cleared - afterwards no Access to pixelserv would be possible

    Please note: This contains pixelserv v34 (you Need to set Access to 0777 for pixelserv) - and script is from jerm slightly modified by me - so not the current exisiting mods or anything like that

    Just tell me if you get stuck - complete documentation and howto will be done on saturday
     

    Attached Files:

    jondlou likes this.
  26. Ravi Baskaran

    Ravi Baskaran Reformed Router Member

    @herrm,
    Thanks for the wonderful script you put together. I'm using your 2014-01-05 script to clock ads on my Linksys E3000.

    I maintain a whitelist. Is it possible for me to restart adblock to update only the whitelist without having to do adblock force?
     
  27. jerrm

    jerrm Network Guru Member

    force shouldn't be necessary for a whitelist update. The script should see the whitelist file has changed and rebuild the list.
     
  28. FameWolf

    FameWolf Serious Server Member

    Thank you jerrm for your jan 5th script....I added it and pixelserv to my belkin N600 tomatousb router and it's working quite well. I stuck with the defaults as far as hosts file but I'm on limited bandwidth so the more ad's I can filter the better so I was wondering what your recommendations of what sources to use would be under those circumstances? I'm using the original config file from post #1 and v34 of pixelserv.
     
  29. jan.n

    jan.n Addicted to LI Member

    Just installed the stuff from the first post, thank you! A question though: Isn't pixelserv supposed to serve a 1x1 pixel?
    I'd say everything's working, because the test URL
    http://rcm.amazon.com/e/cm?t=thedailydownl-20&o=1&p=11&l=ez&f=ifr&f=ifr
    does no longer return a banner.
    Looking at the source in the browser, I see it's completely empty, I would have expected to see the 1x1 pixel instead...
     
  30. lancethepants

    lancethepants Network Guru Member

    You probably did see the 1x1 pixel. Probably just a white dot in a sea of white. It's supposed to be so small that it's inconspicuous. A 1080p monitor has over 2 million pixels.
     
  31. jan.n

    jan.n Addicted to LI Member

    I'm not _that_ stupid ;-) I mean, I can't find it in the html source...
     
  32. jerrm

    jerrm Network Guru Member

    That request is not for a gif image, what pixelsrv returns is a txt/html header with 0 byte content-length and no body.

    In some browsers, if you view source, you'll see some browser generated source for a blank html page, but it did not come from pixelsrv.
     
  33. FretNoize

    FretNoize Reformed Router Member

    Gentlemen, I have a simple question. When I run adblock.sh toggle it disables the adblocker as well as pixelserv. Then when I run adblock.sh toggle a second time, the adblocker is active again, however pixelserv is not re-activated. Is this by design or a bug?

    Thanks

    P.S. Sorry if this seems to be OT and I didn't mean to barge in on this thread but couldn't find the proper place to post and search results did not return good results. Thank you all.
     
    Last edited: Jan 30, 2014
  34. jan.n

    jan.n Addicted to LI Member

    Oh, didn't know either of the two, thank you.

    Anyway, is the source of pixelserv available? The first post links to V3.1, but according to this thread mstombs is already at V3.4... Or is the one from the first post another version?
     
  35. ambiance

    ambiance Serious Server Member

    Edit: NVM
     
    Last edited: Feb 22, 2014
  36. mstombs

    mstombs Network Guru Member

    The latest I posted was simple V34 in that thread a development of V32 , the latest being actively developed has reported VERSION "0.34-2", merges back all changes from V34 and V33 on Github, in particular has an option to
    DECODE_URL built-in (which I haven't got round to understanding!)

    https://github.com/h0tw1r3/pixelserv

    I haven't rebooted my router since I posted V34:-

    Code:
    Oct 14 01:04:18 rtn66u daemon.info pixelserv[13993]: /mnt/usb4gb/pixelserv V34 compiled: Oct 14 2013 00:55:22 from pixelserv34.c
    Oct 14 01:04:18 rtn66u daemon.notice pixelserv[13995]: Listening on 192.168.66.254:80
    Oct 14 01:04:18 rtn66u daemon.notice pixelserv[13995]: Also Listening on 192.168.66.254:443
    ...
    Jan 27 20:30:01 rtn66u daemon.info pixelserv[13995]: 254966 req, 39701 err, 5364 gif, 212 bad, 176825 txt, 88 jpg, 107 png, 122 swf, 32547 ssl
     
  37. FameWolf

    FameWolf Serious Server Member

    Slightly OT but can I run other servers on the ip assigned to pixelserv? Can someone indicate what commands were used to allocate a 2nd ip to the router? Does anyone have a current list of sources? I'm still using the ones in the original post.
     
  38. mstombs

    mstombs Network Guru Member

    I think this is the latest script in this thread by jerrm
    http://www.linksysinfo.org/index.ph...-and-mean-adblocking.68464/page-4#post-237877

    This line creates the secondary IP
    Code:
    ifconfig $BRIDGE:$vif $redirip up
    it will expand to something like
    Code:
    ifconfig br0:0 192.168.1.200 up
    There is alternative syntax to do the same using the "ip" command

    You cannot use the same IP for port 80 or 443, and whether other ports are available depends on what else is using the port and whether its code listens on the interface (all IPs) or just the specific IP - which is what Tomato does for its web gui allowing re-use of 80/443 for pixeleserv
     
    FameWolf likes this.
  39. Scottmsu

    Scottmsu Reformed Router Member

  40. FameWolf

    FameWolf Serious Server Member

  41. vincom

    vincom LI Guru Member

    thnks for this howto, got messed up at first though. because of the formatting.
    got it working flawslessly after a few "dah" mistakes.
    i have a WRT54G-tm do you know how or if i can use its 32mb ram, im using jffs as of now.
    where is the transparent gif located for pixelserv.

    may i make a kindly suggestion, where you have the sentence "if this fails" could you incorporate it into the preceding paragraph, i write quite afew howtos so im just suggesting, i hope im not stepping on any toes, if i did then im sorry as im just trying to help others in my situation
    eg.
     
    Last edited: Feb 8, 2014
  42. ShinichiYao

    ShinichiYao Reformed Router Member

    You can use Privoxy instead.
    Search for "privoxy-blocklist_0.2.sh" to convert adblock+ rules to Privoxy rules.
     
  43. Anubis14

    Anubis14 Addicted to LI Member

    I had an extra bonus with hulu ads being bypassed on Roku, basically the ads would load on the 1x1pixel...until last night. I had changed nothing. Anyone has any ideas how did this work?
     
  44. jochen

    jochen LI Guru Member

    Thank you for that great script!

    This becomes even more valuable since more and more home electronics "phone home".

    One question: do the default blacklists also block trackers, or do they only block ads?
     
  45. ambiance

    ambiance Serious Server Member

    I've updated to jerrm's mod of the script and just have a quick question. On my blacklist I have these entries:

    cdn1.bcnimg.com
    cdn2.bcnimg.com
    cdn3.bcnimg.com
    cdn4.bcnimg.com
    cdn5.bcnimg.com

    Is it necessary to have each entry or would bcimg.com suffice? I know using a traditional hosts file you would have to, but from what I gather this script doesn't work the same way.

    Thanks in advance and thanks for this great script!
     
  46. jerrm

    jerrm Network Guru Member

    For LISTMODE=OPTIMIZE (default) or LISTMODE=LEGACY, bcnimg.com will block all of them.
     
  47. jochen

    jochen LI Guru Member

    I installed pixelserv v34 and jerms script 2014-01-05. pixelserv is listening on port 80 and 443, but https connections fail.
    Code:
    jochen@MacBook:/tmp$ wget https://www.google-analytics.com/ga.jpg
    --2014-02-24 16:38:30--  https://www.google-analytics.com/ga.jpg
    Auflösen des Hostnamen »www.google-analytics.com (www.google-analytics.com)«... 192.168.2.254
    Verbindungsaufbau zu www.google-analytics.com (www.google-analytics.com)|192.168.2.254|:443... verbunden.
    OpenSSL: error:14077419:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert access denied
    Es ist nicht möglich, eine SSL-Verbindung herzustellen.
    
     
  48. jerrm

    jerrm Network Guru Member

    Https connections fail by design, pixelsrv's goal for https is to quickly and relatively cleanly fail the connection during the certificate negotiation phase to prevent retries and timeouts.

    If it were fully ssl enabled and attempted to actually serve content, the user would get browser prompts about certificate errors that would be more annoying and problematic.
     
    Losefrag and darkknight93 like this.
  49. Aubergine

    Aubergine Reformed Router Member

    Love this script! :) Any chance it could get added & built into one of the major firmware mods? @shibby20 @Victek @Toastman
     
    Last edited: Mar 3, 2014
    phuklok1 and Tuurbo like this.
  50. remlei

    remlei Networkin' Nut Member

    if you read the script, it just adds a another interface using a ifconfig.

    Code:
    ifconfig [name_of_the_interface] [new_IP_for_the_interface] [netmask_of_the_current_subnet] up
    sample:
    Code:
    ifconfig br0:lighttpd 192.168.0.253 255.255.255.0 up
     
    FameWolf likes this.
  51. smahnken

    smahnken Reformed Router Member

    First of all, kudos to everyone for all the effort and hard work. I'm looking forward to getting this functional on my N66U.

    Most of my installation seems to be alright, but I think I must have missed a step someplace. It doesn't seem to be blocking ads on Yahoo or Amazon.

    Details:
    - N66U w/Shibby v1.28
    - Script v 2014-01-05, Pixelserv V34. Installed in /jffs/adblock
    - The legacy config file exists (I'm not certain I need it...)
    - Log shows adblock starting, downloading, etc. (I can post if desired). No apparent errors.
    - "ps" shows "3665 nobody 628 S /jffs/adblock/pixelserv 192.168.10.254"
    - Visiting 192.168.10.254 shows a blank page (as expected)
    - No fancy setup (no VPN, etc).
    - For the present, I'm just manually starting it (I.e. I don't have "/jffs/adblock/adblock.sh" in my wanup)

    What can I try next?
     
  52. Mihai Olimpiu-Cristian

    Mihai Olimpiu-Cristian Serious Server Member

    If script works, http://ad.doubleclick.net/ should return a blank page... (If pixelserv runs or an error if it doesn't run, both cases script works).
    If script doesn't work you will get some links...

    If it works (it returns a blank page), probably you need more sources to be enabled for Yahoo and Amazon, but I suspect script it's not working...
    You have to flush DNS on every computer... when you test... or disconnect and reconnect (close the browser).
     
    Last edited: Mar 4, 2014
  53. smahnken

    smahnken Reformed Router Member

    Odd. I thought I had done that, but apparently I was mistaken.

    You're absolutely right. Thank you for the reminder. I needed to clear my FF cache.

    Both FF and IE (which I never use) are now ad-free.

    Thank you!
     
  54. stillsober

    stillsober Reformed Router Member

    complete noob here, can i use this on a 8mb flash router?(belkin n600). Jffs is disabled (read only)
     
  55. remlei

    remlei Networkin' Nut Member

    ^no, you need at least a storage of 1mb space (minimum with the generated adblock-list data is in ram) recommended is at least 2mb (with the default configuration is used) with all blocklist data cached in it.
     
  56. Mihai Olimpiu-Cristian

    Mihai Olimpiu-Cristian Serious Server Member

    Of course you can, you need to use an USB flash drive for storage tough... N600 has at least 1 USB port...
    I provided you with a "light tutorial" on how to get entware going so that should be no problem for you here.
    To install just adjust the prefix in the install script... I don't have too much time to explain but jerrm did a pretty good job in making it simple...
    That is quite wrong...the no part :)
     
    Last edited: Mar 8, 2014
  57. remlei

    remlei Networkin' Nut Member

    ^he doesnt mention anything about USB storage so I just said no with his current question.

    of course you can use USB as a storage option but you dont need a entware or optware to get it working either.
     
  58. stillsober

    stillsober Reformed Router Member

    Oh ok, I think I follow you. What's the PREFIX for the mounted usb stick?
     
  59. Kim K

    Kim K Serious Server Member

    Format a USB stick as ext3, and plug it in.

    Go to the "USB & NAS" then USB Support. At the bottom it will say something like:

    Partition 'sdb' ext3 (14.69 GB / 14.32 GB free) is mounted on /tmp/mnt/sdb

    See Victeks simulator at http://victek.is-a-geek.com/virtual/tomatok26/nas-usb.html

    If you add a command in the "Run after mounting" field like:

    mount -o bind /tmp/mnt/sdb /opt

    It will make it /opt. I believe if you label the partition as OPT then Shibbys build will automatically mount it at /opt.
     
  60. Mihai Olimpiu-Cristian

    Mihai Olimpiu-Cristian Serious Server Member

    @stillsober It's clear you are a beginner at this. Prefix is the USB path where you want to save the script... you can save it wherever you want. Read the installation instructions first.
     
  61. Frequenzy

    Frequenzy Networkin' Nut Member

    im using latest script from jermm, but it seems i can't stop or restart adblock anymore?

    ADBLOCK: Running as /tmp/mnt/sda1/adblock/adblock.sh stop
    ADBLOCK: Another instance found (/var/run/adblock.pid), exiting!
    ADBLOCK: Running as /tmp/mnt/sda1/adblock/adblock.sh restart
    ADBLOCK: Another instance found (/var/run/adblock.pid), exiting!
    ADBLOCK: Running as /tmp/mnt/sda1/adblock/adblock.sh force
    ADBLOCK: Another instance found (/var/run/adblock.pid), exiting!

    -- no choice but to restart
     
    Last edited: Mar 25, 2014
  62. koitsu

    koitsu Network Guru Member

    Make sure adblock isn't running in the process list, then rm /var/run/adblock.pid and try starting it.

    According to the adblock.sh source, force just causes the process to update adblock sources. It doesn't "force a run/launch" of the process.

    Chances are adblock somehow abruptly exited (possibly segfaulted or crashed?) and the pidfile was not removed cleanly. The solution to that problem is to find out how/why this happened. There is no way to "trap/handle" a segfault (if that's what happened), so manual pidfile cleanup is required by the user.
     
  63. jerrm

    jerrm Network Guru Member

    I have seen it hang. Sometimes one or more of the wget processes won't complete and exit, even with the timeout parameter. Haven't bothered tracking down what's really happening. Sort of assuming it's a busbox issue, but really have no clue.

    As koitsu said, check the process list, delete the pid file. Would be curious if wget is the culprit in your case.
     
  64. FameWolf

    FameWolf Serious Server Member

    It's running fine on my belkin n600 but I put the files on /opt/adblock (optware mounted via flashdrive in usb port). If you need more info or specific commands drop me a pm.
     
  65. FameWolf

    FameWolf Serious Server Member

    While it doesn't have enough jffs the belkin n600 sharemax (64mb ram, 8mb flash) runs the script fine if the command is ran from a mounted share such as /opt or /mnt/xxxx (whatever usb drive is connected to the usb port).
     
  66. FameWolf

    FameWolf Serious Server Member

    Whats the latest list of adblock host files? Any new lists? I had to add a few sites to the whitelist to make certain websites available.
     
  67. Mihai Olimpiu-Cristian

    Mihai Olimpiu-Cristian Serious Server Member

    The lists are updated daily, I personally use mvps.org and yoyo.org only, they are enough, remember to schedule the script to run weekly so it can update the lists on it's own...
    And if you see an ad or two it's not the end of the world :) keep it simple keep it fast!

    If you overdo it like adding all lists or adding extra lists that use a compatible format you will end up exactly where you are now, having to unblock certain sites... and also everything will slow down as it will need more and more memory to store all that crap.
     
    Last edited: Mar 26, 2014
  68. Spyros

    Spyros LI Guru Member

    [strike]ADBLOCK: Running as /jffs/adblock/adblock.sh force
    ADBLOCK: Using config file /jffs/adblock/config
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 1:
    : not found
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 10:
    : not found
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 11:
    : not found
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 29:
    : not found
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 30:
    : not found
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 34:
    : not found
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 38:
    : not found
    /jffs/adblock/adblock.sh: /jffs/adblock/config: line 42:
    : not found
    ADBLOCK: Requested list mode is OPTIMIZE
    ADBLOCK: Blocklist active in OPTIMIZE mode
    ADBLOCK: Download starting
    Connecting to pgl.yoyo.org (95.172.9.82:80)
    Connecting to winhelp2002.mvps.org (216.155.126.40:80)
    wget: server returned error: HTTP/1.1 400 Bad Request
    ADBLOCK: Failed: http://winhelp2002.mvps.org/hosts.txt

    Connecting to www.malwaredomainlist.com (143.215.130.61:80)
    ADBLOCK: Downloaded
    ADBLOCK: Generating /jffs/adblock/blocklist - OPTIMIZE mode
    ADBLOCK: Blocklist generated - 7 seconds
    ADBLOCK: 3862 unique hosts to block
    ADBLOCK: pixelserv already running, skipping
    ADBLOCK: Restarting dnsmasq
    .
    Done.
    ADBLOCK: Exiting /jffs/adblock/adblock.sh 0


    Is the host down or should i start from scratch? Link works fine in browser....[/strike]

    Nevermind solved, it was wrong permissions on adblock.sh
     
    Last edited: Apr 2, 2014
  69. godsfshrmn

    godsfshrmn Reformed Router Member

    I am having issues on certain websites where chrome says the page is loading for several minutes. Looking at the dev console when i press stop, various elements (social share buttons mainly) come up that are probably all being blocked by the adblocker. Is there a way to terminate attempts to load them?
     
  70. mstombs

    mstombs Network Guru Member

    Are you using a recent pixelserv with the ssl reply option? This attempts to close https blocked sites quickly which is intended to prevent this sort of thing. But if you use Chrome - you can also use the excellent adblock plus...
     
  71. Jacan

    Jacan Reformed Router Member

  72. jerrm

    jerrm Network Guru Member

    Not for a list like that. You would need a proxy solution to see urls. The script only works at the dns level.
     
    Jacan likes this.
  73. oviano

    oviano Network Newbie Member

    So I came across this thread when looking to block ads using my Tomato router (which is running v1.28.503 Toastman) and I just wondered if here was a step by step guide anywhere to help a newbie to get it up and running?

    It seems the script here is quite advanced - are there big advantages to using it rather than me just following the steps for another (similar?) solution I found here:

    http://www.seiichiro0185.org/blog:2013:09:network-wide_adblock_with_tomatousb

    Many thanks for any guidance!

    Oliver
     
  74. mstombs

    mstombs Network Guru Member

    The script here is basically the same - but collects/merges & optimizes multiple blocklists with whitelists, and it works through dns poisoning. If you only want to use the pgl.yoyo list I'd recommend the dnsmasq domain block version rather than the hosts version.
     
  75. Qvark

    Qvark Network Newbie Member

    I'm experiencing slowdown after having the script enabled for about a day, when I restart it manually the slowdowns disappear. I have a N66U and are using jffs as storage.

    e: ohh, I see. I'm using haarps version. Can anyone tell me how to install the latest pixelserv, can't a handy script like that one in the op.
     
    Last edited: Apr 15, 2014
  76. oviano

    oviano Network Newbie Member

    Many thanks for your thoughts.

    I'll give it a try see how it goes.
     
  77. phuklok1

    phuklok1 Network Guru Member

    This type of functionality is great. While I am using it now (thanks!), I too, hope to see this kind of thing hard coded and added in a new tab within one of the major builds, rather than as a separate script at some point. Thanks again, all.
     
  78. HunterZ

    HunterZ LI Guru Member

    It's a nice thought, but it would likely be impossible to support as a base feature because it requires a significant persistent storage (cifs/USB/jffs/etc.) at least for the blocklist, and even the config files may be a bit large for some nvram limits. It's also a bit dangerous because it's easy for the adblock to overwhelm the memory limits of routers with small amounts of RAM, which would then likely require a hard nvram reset.
     
  79. Qvark

    Qvark Network Newbie Member

    Stupid question: how do I change from OPTIMIZE mode to HOST mode?
     
  80. oviano

    oviano Network Newbie Member

    What sort of hardware does it require - would an E4200v1 be sufficient? Mine says flash size 16MB, total memory 60mb of which 46mb is free.

    If it was built into the UI couldn't it just be greyed out if there was insufficient resources?

    I agree with phuklok1 it would hugely enhance a tomato built to have this built in.
     
  81. Pherp

    Pherp Serious Server Member

    Add the following to your config:
    LISTMODE=HOST
     
  82. Qvark

    Qvark Network Newbie Member

    Thank you! :)
     
  83. HunterZ

    HunterZ LI Guru Member

    That's plenty.

    Personally I only end up using around 5MB for dnsmasq (which does the blocking) with a 1.2MB blocklist, but that's probably smaller than most people's because I only use lists that report last-modified times so that the blocklist is not regenerated every time I check for updates.

    No, because the required amount of resources cannot be determined ahead of time: It's entirely dependent on the cumulative size of whatever host lists you configure it to download and process.
     
  84. oviano

    oviano Network Newbie Member

    Great, thankyou.

    Maybe it could use a proportion of the memory available and then if it runs out beyond that log an error or something?
     
  85. phuklok1

    phuklok1 Network Guru Member

    I'm running it on a decade old wrt-54g v2 (what an amazing workhorse it has been thanks to the community) which is probably close to the lowest end specs still around (and next to an old 80s model M keyboard, probably the oldest thing I have in continuous use, LOL), so as long as you don't go nuts with lists, I think it should be possible to have it built in with some smart safety checks of course.

    Also, correct me if I'm wrong, but having it hardcoded would reduce the amount of NVRAM used since we wouldn't be storing the associated adblocking scripts there.
     
    Last edited: Apr 19, 2014
  86. HunterZ

    HunterZ LI Guru Member

    Something about which lists to use would have to be stored in nvram or other persistent storage, along with white- and blacklists.

    I hacked things up on my local copy so that the whole thing runs from cifs, including the script itself. This made it easier for me to tweak the script as needed.
     
  87. timjordan

    timjordan Connected Client Member

    Could I ask for a step by step to install this script. One that mere mortals can comprehend. you know the telly tubby version. I have tomato running on my Cisco e3200. Works great, would like to add the ad blocking and the power that this script promises. The instructions gloss over some of the points as if they are common knowledge. They are not. Anyone interested in writing out a "one two three" version of this scripts implementation that assumes very basic knowledge about routers? I have about 15 years experience in the repair of computers, troubleshooting windows and large format printing repair.
    I am 50 years old. However some of the terms used in the description were unknown to me.
     
    Last edited: Apr 20, 2014
  88. HunterZ

    HunterZ LI Guru Member

    The instructions in the first post were clear enough for me, so you're going to have to ask specific questions about the parts that aren't clear enough to you.

    There are a lot of options on how to set it up that are covered, so you'll have to make decisions about what you want to do (for example, do you want to use cifs, jffs, or USB? Do you want to use pixelserv or not? etc.).
     
  89. timjordan

    timjordan Connected Client Member

    well take for instance xcoolings instructions


    Installation Instructions:
    I am glad the instructions were clear to you. however I am needing them clarified.

    Check out the instructions for x cooling for a contrast.
    1. Go to your Tomato Web Interface ( default: http://192.168.1.1 )
    2. Navigate: Administration -> Scripts -> WAN Up
    3. Copy the contents of the box below and paste it into the empty area (you do not have to chnage anything)
    4. Click the SAVE button at the bottom of the box.
    5. To enable the script: restart the router or disconnect and reconnect the WAN/internet connection.
    6. To test: navigate to a website with lots of adverts. (eg: http://pagead2.googlesyndication.com )

      I clipped the script, intalled it and they worked right away.
    compare the instructions for this script at the first post.

    • Designate a directory on your storage for adblock, e.g. /jffs/adblock/ (as seen by the router). Avoid spaces! This is the PREFIX.
    • Install pixelserv if desired (thread). Take the entire chunk of script in this link, adjust PREFIX at the top, paste it into the box on Tools->System and press Execute. This is also how you can update pixelserv in the future.

      My thoughts:
      where do you designate the storage for adblock? is this on the computers i am using or is there some sort of storage on my router? I am assuming there is no storage on my router but the ram. Is this storage on a connected usb hard drive? I am using a mac, do these instructions cover that or are they for a pc? is there a link with the instructions for ad block? I am familiar with ad block on chromium, firefox, chome etc. Is this a separate ad block? If so is there a link to it? If you look up ad block there are many links that come up.

     
  90. timjordan

    timjordan Connected Client Member

    "adjust the prefix." what is that? where is the prefix, how is it adjusted, adjusted to what?
     
  91. timjordan

    timjordan Connected Client Member

    "Install adblock.sh v4.5. Take the entire chunk of script in this link, adjust PREFIX at the top, paste it into the box on Tools->System and press Execute. This is also how you can update Adblock in the future."

    when i clip the chunk of script from the link above and post it in the tools>system and then press execute I get this.
    mkdir: can't create directory '/cifs1/adblock/': Read-only file system
    /tmp/.wxzBxUU8: line 53: can't create /cifs1/adblock//adblock.sh: nonexistent directory
    chmod: /cifs1/adblock//adblock.sh: No such file or directory
    SO I need to create /cifs1/adblock//adblock.sh:

    where is this to be created? The instructions are missing some step to make it work.

    I did not adjust the prefix at the top because I didnt see any instructions on how to adjust it.
     
  92. Spyros

    Spyros LI Guru Member

    if you have enabled jffs then it should be

    Code:
    #!/bin/sh
    ## install adblock v4.5
    ## encoded: cat adblock.sh | gzip -c | openssl enc -e -base64
    
    PREFIX="/jffs/adblock/"    ## adjust this!
    
    [ -d "$PREFIX" ] || mkdir -p "$PREFIX"
    echo "
    H4sIAFsj3FECA5VXf0/bSBP+O/4Ug7FwUsVxAlfpZC7RS4GW6GjDC0F3Utt75cSb
    ZA9nbbw2obry3d+Z3XXsBFJ6QsTe9czss7PP/Nj9PX/ChS8X1v4+nMYsFG24xF8I
    RQQf6eUkmsTJ9A4efum8hck3WIRhlqI0KYzSnCdCBvTuzpJsytwA1BOKNApzLuYg
    kwLHsq1kZJ6kKBJxGU5iVtpug5sn83lM2vcFn97F30BPrFdPNKJkNlN2MibzMMtR
    3rytBZusM+8QBpgmYsbnMF2EYs5kyyLF8ehsFIA7pY26bcIT5oV0IdEbsaww5qEE
    Fifzvos/c5aBh8bP3l2OTn8HT7pGgqc57UD2Xb+QmS/JieWca1kpj2Y8Zn3/Icz8
    rBB+qOF18IN1x+MYvC44zWmYg2Nk4XDgR+zBF0Uct+CgGsDBAfxjNQgT2CciyRcI
    iguELtDPs6RAvzRLK602sEdOjt+zUQdfoWc9WWy6SMBxYLBeDjHS12arsn2uFVEv
    W1Zy2ojzH7Ri0fHVNW5wnK5VbOd09Om9XcduWY3SK+CdwfDT1e0YPA7Ou+vh2Ydz
    8FLIpyl4ETgZi3jG8d2L0gQP9NcueH/Dyenp+dV4w2SD/BeiX1L+yGLJsofNz3xm
    Tt4sEvQgSlZiS+gVVGR/A9XfcHY9utram/bCWSJYuyQiUT4SchnKe/QKoeN4Smam
    FCJfzrNwomNDexTpEcawSGTed5r6vHrwHcLVHXjvbd8G95804wKP4ujJbZXyaZgv
    XpGXxUTmWdPptpE1EXukN+eXVqtmJQ5lvkyi0pDHwL44PzkD3yH7cDEeX/m9Tu9L
    9kVcIMAAHMJJQ/q3cVkxBW911NUf6Oy+Q56RB90vmYuDecZSsC9xHe9jEvEZZ5GN
    IVmuv1TRYjsxl3masRl/9A0mr765ZfRWFkt8mRY5eFPoeb+27NKIduZzQ3r+J+xY
    jc/IYZW+bNjrg92zwQuVN5zKuJ4TOGcQqonaqI+aOrRxcqlV6rFtw1cT0oY/t0Ln
    qCggdM3SEu2soTAfwmCwuSWVtuh7xvIiE3BoNZBSjSbO8Bms5iwnU94IvGPAfCFw
    Hje3iVqhUPZrk4MKNOmoz93dy1P8KTm1kfch6qldVMq93cozbjVaa6LYXzIikmQR
    edz1//r8OQhjUSyDr8FXfy9yDbnXYXD45G7IKxYQ/RwfhQebh0YRR2lhzkQ9gX1g
    gmW6TtUxqlxN4x8mthl81uQwWqsFz5nSgq9rt0vKZV4BdR/YJSfflJHhPYA3e9HS
    YENzA5lx/isr/MAAHQBVyhUmBnD+uBiOzy+HN+NjzJdklzzL1Q59Z+UjOXYaijAD
    qvCpu2MSh0agIryOiucCgx+gfNIgJwrku8uT099rIA1/Jz82YfBVO0K6KJYQXhn8
    FYRRhKlZ9v3ANnNO4JfJP9i985JHp7rizDWdWNTG4r7ClBjDb7t0W1AIfl8wlfIl
    9jugvimmauE+pZGIkgnCEOGSfNu1MX+gM9MV5YdNh+uyR97+/r2i+OZXwP5B9wzU
    H6S6QaAOQS1kyNPYZ0IWGUMO43kZxQV2PchpHGYZm+aq8liaZWCbpS1LBYXtXJ98
    pEPSybBXi4fKFX1bNUcxn5TNka0YFG0SWe9meYcu2PxgafrXDZb4LSS2AXI1/PP8
    8n/DK42kW0NiDrdvdzvqrzSIEB4rr617jMqtW36tupC1a332yLCyUINBXi7d3FVc
    Ltd1mtuNSpngfC4wfxMl/VrJr2p7YO9Igrb0D9++9deb9rHEoisw84WSeIIHwdXO
    VRPSAmrnjo8xLvDZUr/HoLF2aVq34K0yrHUO1KEMm8LwRPKqcrb0o4+nTnO23WoE
    9PKmZfzmOj1XuQodxTK0kyAVS1M9FGUynCLmsrdaiTgJIyibK9sqRM6p+cOc7d1j
    9e7BPEkQaGeaLKHK0DpBaCt/hKqzLa8FAvnLH3j+rdPpUJMWM0zBR11Lp4k8C1Nw
    TYPL5wJbCyQ84w8U1lNquuOYgByDKg0vJt2DOo5nYmVrsyVnXHDUdeFm+OG/t8Mx
    PYef1GN8fv2Rnhe3V5Zu56maY9p7sbjSRqVKmDej2+vT8xvtjarnBEfCgd7wCp1j
    WVhy8A6SMXQ19hRa3NAGx5o4jW6rEZkDYVGfDqvR6OGJq7pvxoetRlE2NHpKHSgt
    tZ0MS7RbDlrP66PwXvOGCfUK2fO0s0kmlMDN6W4AeWb016h3qb/ncY6MhUrQNAF7
    OkBeTPO1VqBcsWGW3NuIKy0G+4KxCPl1hwqBrvGGGIeqYOsktRUc+gDsek/yMhjs
    VP/lqkdqVcoiutTSJjzVZO9YYlAat57n4L2tHIwCqaz1QOpZDu9fyrFlTbbXXtWu
    qCTCmEj8DfDWLTBM2yDveHlJNR2TiW2Wq6RQpLWbZCKqJSzqptf3xIdLc1Gs49OJ
    u/NmfU98Q9fZKM2D1WpllwWjZmX4b26bP6P6M9dntPOkbgbPLsZrtSJFgZeq3lrC
    HOPoanxTUeKV+++u669lCof1f4rwgu1/EgAA
    " | openssl enc -d -base64 | gunzip -c > "$PREFIX/adblock.sh" && echo success
    chmod +x "$PREFIX/adblock.sh"
     
  93. timjordan

    timjordan Connected Client Member

    Wow, thanks so much! I looked up jffs and then formatted it, worked really well. That is pretty amazing to have space on your router where you can store programs like optware, really great. My kids like running a minecraft server, going to see if that's possible with the optware. I really do not know how much space it has there, going to research it thanks for the help!
     
  94. timjordan

    timjordan Connected Client Member

    says I have 6,016.00 KB / 5,688.00 KB so thats a lot more than I would have guessed.
     
  95. HunterZ

    HunterZ LI Guru Member

    Glad you got it working.

    Only downside of jffs is that flash storage has limited write cycles before it wears out. I use a cifs (samba) mount to an always-on Linux box on my LAN. A third (and possibly best) option is to use a USB stick if your router has USB ports.

    Also, you may want to look into Entware as an alternative to Optware. I'm not sure how or why one might be better than the other, though.
     
  96. phuklok1

    phuklok1 Network Guru Member


    Of couse it's true that the desired lists would need to be in nvram if ad blocking was built-in, but the actual list data could simply remain in RAM if desired which has the added bonus of not constanltly rewriting the static ram and not requiring jffs or cifs for those with more modest setups or routers. With the super long uptimes you get with tomato, losing a few minutes rebuilding the list data in the rare event the router loses power or needs an upgrade is not really a big deal. IMO, the advantages in setup, ease-of-use and management for such a popular feature would be huge. We have build options for so many less used features, this one would seem like a another great addition and enhance the default flexibility of Tomato. I hope shibby, toastman, victek or one of the other great builds picks it up eventually.
     
  97. HunterZ

    HunterZ LI Guru Member

    Yeah, you could probably write it to a file on a part of the router's filesystem that is stored in RAM at run-time. The downside is that the RAM requirements would then be an even higher multiple of the blocklist size.

    For what it's worth, my blocklist (which is probably on the small but effective side) regenerates in 10-15 seconds on my RT-N66U. I believe that the same list took around 90 seconds on my WRT54G, and maybe something like 30-40 on my RT-N16.
     
  98. phuklok1

    phuklok1 Network Guru Member

    btw, the issues people are having in other threads with precompiled binaries like pixelsrv or others on various platforms, kernels, builds, etc is a case in point as to why having this feature compiled in to each specific build as an option would be of such a large benefit to the general tomato community.
     
  99. jerrm

    jerrm Network Guru Member

    I have mixed feelings about incorporating it into the builds. It sounds like a nice idea, but I hate the thought of the devs spending time on it. Certainly shouldn't be considered until we have an otherwise stable fully functional ARM platform.

    Although it could be greatly simplified if built-in, if the ability to specify various source lists and add white/blacklist support is enabled, there would still be an impact on NVRAM of several hundred bytes.
     
  100. iommi

    iommi Network Newbie Member

    jerrm,

    I have your script installed an running (edit: problem also occurs using the original haarp script v4.5); however, I have had problems with some of the host lists that are provided in the default config. Here is an example:

    ADBLOCK: Running as /cifs1/adblock/adblock.sh
    ADBLOCK: Using config file /cifs1/adblock/config
    : not found /cifs1/adblock/config: line 9:
    : not found /cifs1/adblock/config: line 27:
    : not found /cifs1/adblock/config: line 31:
    : not found /cifs1/adblock/config: line 35:
    ADBLOCK: Requested list mode is OPTIMIZE
    ADBLOCK: Blocklist active in OPTIMIZE mode
    ADBLOCK: Config or script has changed - rebuilding list
    ADBLOCK: Download starting
    Connecting to winhelp2002.mvps.org (216.155.126.40:80)
    wget: server returned error: HTTP/1.1 400 Bad Request
    ADBLOCK: Failed: http://winhelp2002.mvps.org/hosts.txt

    ADBLOCK: Processing blacklist only
    ADBLOCK: Generating /cifs1/adblock/blocklist - OPTIMIZE mode
    ADBLOCK: Blocklist generated - 0 seconds
    ADBLOCK: 4 unique hosts to block
    ADBLOCK: Setting up 10.0.0.254 on br0:adblk
    ADBLOCK: Setting up pixelserv on 10.0.0.254
    pixelserv[7280]: /cifs1/adblock/pixelserv V34 compiled: Oct 14 2013 00:55:22 from pixelserv34.c
    ADBLOCK: Restarting dnsmasq
    .
    Done.
    ADBLOCK: Exiting /cifs1/adblock/adblock.sh 0

    It looks like wget is having a problem when run with your script (I am using adblock-2014-01-05). I can run wget from a command line and the file downloads fine:

    root@tomato:/cifs1/adblock# wget http://winhelp2002.mvps.org/hosts.txt -O test
    Connecting to winhelp2002.mvps.org (216.155.126.40:80)
    test 100% |*******************************| 499k 0:00:00 ETA
    root@tomato:/cifs1/adblock# ls test
    test

    Do you have any ideas what the problem may be? I am running Shibby's Tomato v119 AIO on ASUS RT-N66U.
     
    Last edited: Jun 1, 2014

Share This Page