1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Script: Clean, Lean and Mean Adblocking

Discussion in 'Tomato Firmware' started by haarp, Apr 23, 2013.

  1. Almaz

    Almaz Serious Server Member


    That's right. It's taking from all the log files which could be multiples.
    /tmp/var/log/messages
    /tmp/var/log/messages.0

    dnsmasqlog="/tmp/var/log/messages*"
     
  2. Almaz

    Almaz Serious Server Member

    In my case, here is a fix for it. Will it work for you?

    Code:
    egrep -B1 "config .* is $pixelservip" $dnsmasqlog | egrep 'query.* from ' | grep -v 'from 127.0.0.1' | tail -n 100 | sed 's|^\(.*:..:..\) .*: quer|\1 |' | awk '{printf("%s %s %s) %-13s %s\n", $1,$2,$3,$7,$5)}' | sed -r 's:^/tmp/var/log/messages(.0)*-::' | sed 's/[)]//'
     
  3. HunterZ

    HunterZ LI Guru Member

    Ah. I decided not to spam up my main syslog with query data, so I have dnsmasq set to log queries to a separate file.
     
  4. Almaz

    Almaz Serious Server Member

    Probably the best way to do it, if log is in internal place then use below code, else to use yours.

    Code:
    egrep -B1 "config .* is $pixelservip" $dnsmasqlog | egrep 'query.* from ' | grep -v 'from 127.0.0.1' | tail -n 100 | sed 's|^\(.*:..:..\) .*: quer|\1 |' | awk '{printf("%s %s %s) %-13s %s\n", $1,$2,$3,$7,$5)}' | sed -r 's:^/tmp/var/log/messages(.0)*-::' | sed 's/[)]//'
     
  5. Nathaniel Cowles

    Nathaniel Cowles Reformed Router Member

    I had to take a short hiatus to figure if/why my GUI WAN Up script is not running. It's running sometimes and I don't know why. I have my dnsmasq.log in /mnt/usb/var/log/
    and it rotates, have AndreDVJ's adblockWebControl.sh linking and running okay but not quite right. The hosts total usually shows !!Restart!! and restarting doesn't fix it.
     
  6. Almaz

    Almaz Serious Server Member


    If restart doesn't fix it then you have to make sure adblock.sh have write access. Just chmod 777 and see if it helps. My script have to make an injection to adblock.sh and modify it.
     
  7. HunterZ

    HunterZ LI Guru Member

    Why does it modify adblock.sh? That's scary!
     
  8. Nathaniel Cowles

    Nathaniel Cowles Reformed Router Member

    Both my adblock.sh and adblockWebControl.sh are chmod 777. I have seen the hosts total there but no matter what I do now it says !!Restart!!

    I think I will wait a few days for the control script to be refined. Ive re-downloaded adblock.sh and cleared adblockWebControl.sh for now.
     
  9. Nathaniel Cowles

    Nathaniel Cowles Reformed Router Member

    Thanks much! That was perfect to get me going.
     
  10. rs232

    rs232 Network Guru Member

    Have used this script for sometime. Thanks a lot, it works brilliantly!
    If I can give an input for improvement, it would be great to have a trivial in-line editor (like a mini ajax script) to allow LAN users to add their whitelisted domains into the whitelist file.

    my 2 cents
     
  11. HunterZ

    HunterZ LI Guru Member

    I thought about that too, but it's beyond my abilities since I'm not a web developer.
     
  12. Almaz

    Almaz Serious Server Member


    The script works without a glitch. The reason it doesn't work because you didn't read properly my directions from my post. You can't just copy and paste the script, and expect it to work. The first 5 or 6 lines must be configured for your setup. If you don't configure them properly then you can't expect it to work.
     
  13. Almaz

    Almaz Serious Server Member

    Nothing serious, it just creates a log file to count hosts names. There are other way to do it without an injection but I don't see any reason to reinvent the wheel.
     
  14. HunterZ

    HunterZ LI Guru Member

    A word of warning:

    I have dnsmasq set to log queries on a cifs mount. When I rebooted the cifs host, dnsmasq freaked out and stopped being able to function. This in turn resulted in my cifs host not being able to regain network connectivity since dnsmasq wouldn't hand out DHCP info (I didn't think to set the cifs host to a static IP, which may have worked), which in turn continued to prevent dnsmasq from being able to recover.

    The fix I ultimately employed was to temporarily modify the dnsmasq custom config in the router GUI to disable logging of queries to the cifs mount, after which the cifs host was able to regain connectivity via DHCP.

    I should really think about getting a small USB stick to plug into the back of my router for non-volatile storage, instead of using cifs. I suppose jffs would work too, but I'm just too paranoid for that :p
     
    koitsu likes this.
  15. AndreDVJ

    AndreDVJ Addicted to LI Member

    I have a 64GB SanDisk "nano" stick. I don't see myself running out of space on my router soon.
     
  16. Nathaniel Cowles

    Nathaniel Cowles Reformed Router Member

    My apologies Almaz. Indeed I did configure the first lines of the script to work properly for my set up. I will certainly have another go at it.
     
  17. AndreDVJ

    AndreDVJ Addicted to LI Member

    Hello all, I got a very weird "issue" with HTTPS sites (at least https://www.facebook.com) and I'll try my best to explain. I didn't think a new thread was needed.

    I run Windows 7 x64 and I decided to look at the event log for errors.

    Running pixelserv V34, I found many entries with "Schannel" as the source. Event ID is 36887. The details say:
    The following fatal alert was received: 49.

    Well, it was spamming my event log since May, so it's not really new, I haven't checked logs for months.

    I started to Google around and all I saw was issues with CA and Exchange Servers (This is my desktop, I don't run any server on it).

    What I could figure out at that moment is that stopping the script (and pixelserv as consequence) "cleared the issue".

    Then I started to test around with browsers and IE11 triggers the error (I mainly use Internet Explorer). Google won't trigger the error. At least I was able to reproduce by logging on Facebook (has plenty of ads to kill).

    Then I started to look for an updated version of pixelserv and compiled the source code on my router (gcc -o pixelserv pixelserv.c) (Thanks @lancethepants for tomatoware).

    Well, compiled just fine.
    Code:
    pixelserv[23851]: /mnt/storage/adblock/pixelserv version: 0.34-2 compiled: Aug 20 2014 21:46:01 from pixelserv.c
    And I got a different error. Event ID 36888
    The following fatal alert was generated: 10. The internal error state is 10.

    That drove me nut. I started Googling around again and found a workaround, to stop Schannel spam on the system log:
    http://www.petenetlive.com/KB/Article/0000634.htm
    And I disabled Schannel logging as instructed. Yeah that cut off the bleeding.

    Keep Googling, I stumbled across a good article, with a compilation of the TLS error codes:
    http://blogs.msdn.com/b/kaushal/archive/2012/10/06/ssl-tls-alert-protocol-amp-the-alert-codes.aspx

    Well I'm not good handling certificates nor I do have proper understanding of the TLS protocol. You might think I am lazy by not reading the RFC regarding the protocol.

    Now I have a couple of questions... Since IE is much tightly integrated with the Windows OS itself (obviously, because it's a M$ product), I am speculating that since the browser did not receive the page and something else (pixelserv) sent the ad of the website running the ad, the OS found it odd and logged the error.

    Is my assumption correct? If I am correct, was this the expected behavior?
    How two different versions of pixelserv would send a different response?

    I didn't test many websites to confirm to be honest. I'm more just reporting something than finding a "solution" because I honestly don't think it has any.
     
    lancethepants likes this.
  18. Almaz

    Almaz Serious Server Member

    Last edited: Aug 21, 2014
  19. mmosoll

    mmosoll Serious Server Member

    Only an idea or alternative for all users of adblock.sh having problems with pixelserv: use the NGINX server embeded in many last Tomato firmwares distributions (I am using TomatoRAF v1.28.9014 v1.3f and e_4200 router).

    I used the following tutorial:
    http://www.shadowandy.net/2014/04/adblocking-nginx-serving-1-pixel-gif-204-content.htm

    1. Deactivate pixelserv in your config file used by adblock.sh

    2. Run NGINX server in Tomato

    nginx_config.png

    In the NGINX Custom configuration textbox put the following parameters (the previous configuration link reference is a little bit modified). I included an http/https configuration (you must change ssl certificate if you want to use https) and eliminated NGINX access and error logging. I'm only a beginner with NGINX, check the parameters before use.

    Code:
    # NGINX Custom Parameters.
    # AdBlock Entries - http://tiny.cc/nginx-adblock
    http {
        log_format main '$remote_addr - $remote_user [$time_local]  $status '
    '"$request" $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';
        server {
            listen 80;
            server_name adblock;
            access_log /tmp/var/log/nginx/access.log main;
            access_log off;
            error_log /dev/null crit;
            expires max; # instructs browser to cache the reply
            rewrite .+?(png|gif|jpe?g)$ /empty_gif last; # redirect image requests
            rewrite ^(.*)$ / last; # redirect all other misc requests
            location / {
                 # return 204; # return no content
                 empty_gif; # serving 1 pixel gif
            }
            location /empty_gif {
                empty_gif; # serving 1 pixel gif
            }
        }
        server {
            listen 443 ssl;
            server_name adbock;
            # If you have a certificate that is shared amongst several servers, you
            # can move these outside the server block.
            ssl_certificate /tmp/mnt/div/ssl/vspd.pem;
            ssl_certificate_key /tmp/mnt/div/ssl/private/vspd.pem;
            access_log /tmp/var/log/nginx/access.log main;
            access_log off;
            error_log /dev/null crit;
            expires max; # instructs browser to cache the reply
            rewrite .+?(png|gif|jpe?g)$ /empty_gif last; # redirect image requests
            rewrite ^(.*)$ / last; # redirect all other misc requests
            location / {
                 # return 204; # return no content
                 empty_gif; # serving 1 pixel gif
            }
            location /empty_gif {
                empty_gif; # serving 1 pixel gif
            }
        }
    }
    # End of AdBlock Entries
     
    Last edited: Aug 21, 2014
    Losefrag likes this.
  20. mstombs

    mstombs Network Guru Member

    Interesting 'unexpected consequence' I will investigate if I also have such windows error logs

    pixelserv.c sends that "TLS access denied" code when compiled with -DSSL_RESP

    Code:
    #ifdef SSL_RESP
    static unsigned char SSL_no[] = "\x15" // Alert 21
    "\3\0" // Version 3.0
    "\0\2" // length 2
    "\2" // fatal
    "\x31"; // 0 close notify, 0x28 Handshake failure 40, 0x31 TLS access denied 49
    #endif
    I did experiment with the other replies, but then the browser tended to retry... No attempt to open a communication channel is made, only intent is to close the connection quickly hopefully avoiding page load hangs.

    I think you compiled without SSL_RESP (and many other options?) so the browser received a default GIF or text message. I don't think V35 will make much difference, but for me it fixes a jpg load error.

    http://www.linksysinfo.org/index.ph...run-on-router-wrt54g.30509/page-3#post-242718

    Maybe the above config using a real web server NGINX that can talk https will help.

    Many more adverts are being served over https connections, but likely they are rarely simple images, and by definition of secure connection it should not be possible for the router to know what is being asked for - without being flagged as a Man-In-the-Middle hacker!
     
  21. Beast

    Beast Network Guru Member

    I am seeing the same error using IE on Win7 64. (The following fatal alert was received: 49.) But all seems fine using FireFox. Cleared system Log and Loaded up IE clicked weather TAB, and looked at system log again, and got a about 8 alert messages.
     
  22. AndreDVJ

    AndreDVJ Addicted to LI Member

    @mmosoll I gotta try that method someday. The script is blocking ads, working as intended though I didn't expect that side effect. It's not really a problem, that's the reason I referred as an "issue" inside double quotes because I don't and didn't know how to word better than this (I don't speak english natively).

    @Beast Yeah it looks like only happens with Internet Explorer.

    @mstombs I compiled pixelserv.c straight from the router (gcc -o), so I didn't think (wasn't aware) of these options because I didn't have a build script at hand. I downloaded V35 from the link you provided and found the options you mentioned. The build script won't work on the router because I don't have libraries mipsel-uclibc-gcc and mipsel-uclibc-strip installed. I'll eventually try to get this built properly as a challenge.
    In shoort words, whatever I compiled wasn't working as intended.

    For the error codes, the portion of the source code you posted clearly says it's intended to work that way.

    The only "issue" is that Internet Explorer floods Event Logs whenever pixelserv sends the message to the browser. I got it covered already, taking down Schannel logging.
     
  23. lancethepants

    lancethepants Network Guru Member

    @AndreDVJ
    Since you're natively compiling on the router, the way you have done it is correct. mipsel-linux-gcc is for cross-compiling on x86 machine. You can look at the Makefile to see a list of options (OPTS) you can include.
    https://github.com/h0tw1r3/pixelserv/blob/master/Makefile

    You could also throw in '-static', and that will make a portable binary that can be used on systems that don't have tomatoware.
    Running
    Code:
    readelf -d ./pixelserv
    
    Should then return the following.
    Running
    Code:
    strip ./pixelserv
    
    Might reduce the file size a bit too.

    Somehow the author has posted some ridiculously small binaries on github, it almost makes me think they wouldn't work, being so small (I haven't tested them), even if they've also been compressed with upx. The TomatoUSB toolchain (cross-compiling only) actually produces smaller binaries, but the tomatoware environment is much more up-to-date comparatively.
     
    Last edited: Aug 21, 2014
  24. AndreDVJ

    AndreDVJ Addicted to LI Member

    I figured out and thanks lancethepants.

    Replaced few lines on the build script to just call 'gcc' instead of 'mipsel-uclibc-gcc', and just 'strip' instead of 'mipsel-uclibc-strip'. Also corrected export location to /opt/tomatoware/

    Compiling with -static gave the same binary size. The only dependency the binary has for both cases is libc.so.0. The one I downloaded also requires libgcc_s.so.1 so one less depedency, even though both are present on the tomato firmware. Maybe will perform better, since I compiled on the architecture it's going to run.

    Compiling from the router reduces size of the pixelserv binary from 10.324 bytes to 9.332 bytes (1KB)

    So far I'm running what I have compiled myself and it's blocking ads as intended. I still get the same error events, though I don't care about them anymore. I just turn them off in registry.
     
  25. mstombs

    mstombs Network Guru Member

    I don't often use IE (Firefox and Chrome also have excellent adblock plus!), but have now tried and also see the log errors. I doubt if any other alert code would help (code change and recompile?) - also might be better to use iptables to REJECT the https connection (DROP no good, it will lead to time-outs and retries) using the script. Static pixelserv binary usually many times bigger, lots of library calls but all ancient functions so doubt any benefit in later toolchains, but always best to to use the right one for the firmware! Pixelserv binary zip compresses to half size, but I never saw much benefit in upx compression.
     
  26. koitsu

    koitsu Network Guru Member

    Regarding http and https connections: assuming the iptables rules are written specifically with TCP in mind (-p tcp), what should be used is --reject-with tcp-reset (which sends a TCP RST back to the client immediately; no ICMP is used). Browsers will understand that quickly (regardless of HTTP or HTTPS). However you might get weird/wonky error messages when using HTTPS because of the SSL encryption, but the browser should notice immediately.
     
  27. HunterZ

    HunterZ LI Guru Member

    Yeah, I used to use a --reject-with tcp-reset iptables rule for https before someone made pixelserv more https-friendly. I think the iptables method works at least as well as letting it go through to the newer pixelserv builds.
     
  28. leandroong

    leandroong Addicted to LI Member

    Thanks. Installed on my RT-N56U, padavan FW using tomatoware as binary compiler.
     
    Last edited: Aug 26, 2014
  29. leandroong

    leandroong Addicted to LI Member

    ## Clean, Lean and Mean Adblock v4.5 by haarp
    During update, " sh adblcok.sh force", I notice ff error :
    Code:
    /media/optware/adblock # sh adblock.sh force
    ADBLOCK: Download starting
    adblock.sh: line 136: nc: not found
    adblock.sh: line 136: nc: not found
    adblock.sh: line 136: nc: not found
    --2014-08-26 17:49:17--  http://www.malwaredomainlist.com/hostslist/hosts.txt
    Resolving www.malwaredomainlist.com... --2014-08-26 17:49:18--  http://winhelp2002.mvps.org/hosts.txt
    Resolving winhelp2002.mvps.org... --2014-08-26 17:49:18--  http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
    Resolving pgl.yoyo.org... 80.87.131.59
    Connecting to pgl.yoyo.org|80.87.131.59|:80... 143.215.130.61
    Connecting to www.malwaredomainlist.com|143.215.130.61|:80... 216.155.126.40
    Connecting to winhelp2002.mvps.org|216.155.126.40|:80... connected.
    HTTP request sent, awaiting response... connected.
    HTTP request sent, awaiting response... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 40851 (40K) [text/plain]
    Saving to: 'STDOUT'
    
    0% [                                       ] 0           --.-K/s              200 OK
    Length: 511276 (499K) [text/plain]
    Saving to: 'STDOUT'
    
    0% [                                       ] 0           --.-K/s              200 OK
    Length: unspecified [text/plain]
    Saving to: 'STDOUT'
    
    100%[======================================>] 40,851      76.2KB/s   in 0.5s
    
    2014-08-26 17:49:19 (76.2 KB/s) - written to stdout [40851/40851]
    
        [   <=>                                 ] 66,178      87.9KB/s   in 0.7s
    
    2014-08-26 17:49:19 (87.9 KB/s) - written to stdout [66178]
    
    100%[======================================>] 511,276      204KB/s   in 2.4s
    
    2014-08-26 17:49:21 (204 KB/s) - written to stdout [511276/511276]
    
    ADBLOCK: Downloaded
    ADBLOCK: Generating /media/optware/adblock/blocklist
    ADBLOCK: Config generated, 16814 unique hosts to block
    ADBLOCK: Setting up pixelserv on 10.0.1.254
    pixelserv[1220]: /media/optware/adblock/pixelserv V35 compiled: Aug 26 2014 13:07:58 from pixelserv35.c
    ADBLOCK: Done, restarting dnsmasq
    
    adblock.sh: line 136: nc: not found ?
     
  30. Almaz

    Almaz Serious Server Member

    Before you start doing anything restart your router.
    ssh into your router and type "nc" then press enter.
    type "which nc" press enter
    type "df -h" press enter.
    type "echo $PATH"
    post your output here.

    check if adblock still showing the same error after restart. If it's still not working then redownload your adblock.sh and use only one host for ads instead of multiple ones. Post your results.
     
    Last edited: Aug 26, 2014
  31. leandroong

    leandroong Addicted to LI Member

    padavan FW does not have function "nc". Is there an optware nc?
     
  32. Almaz

    Almaz Serious Server Member

  33. leandroong

    leandroong Addicted to LI Member

  34. leandroong

    leandroong Addicted to LI Member

    I just learned that "nc" and "netcat" are the same. I replaced "nc" with "netcat" in adblock.sh, result is the same.
    Note: netcat is entware optware
     
  35. leandroong

    leandroong Addicted to LI Member

  36. Almaz

    Almaz Serious Server Member

    In Chrome browser, right click on ad and press on Inspect element. Add host name to black list. FYI I don't see any ads
     
  37. leandroong

    leandroong Addicted to LI Member

    There is initial popup ad from left bottom, then it will disappear and never have new popup ads. Refreshing will show initial popup ads again.
    This is good enough for me, looks perfect.
     
  38. Drats

    Drats Network Newbie Member

    Is anybody else having problems downloading the yoyo list in ADBLOCK?

    If I enable SOURCES="$SOURCES http://pgl.yoyo.org/as/serverlist.php?hostformat=nohtml" the resulting source file is just blank lines. Looking at the raw file from yoyo.org, it looks like it is already in the proper format for the script to process into a blocklist without the manipulations in grabsource().

    Ray
     
  39. HunterZ

    HunterZ LI Guru Member

    Not sure if it matters, but I've been using:
    SOURCES="$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext"

    Not sure where I got that from. Someone must have mentioned it earlier in the thread?
     
    Drats likes this.
  40. Drats

    Drats Network Newbie Member

    That looks doable. I use another file for DD-WRT, but it needed work also, I will try this one when i get the chance. Thanks!
    This seems to be working pretty well on my RT-N16 using Merlin 376.45 .

    Ray
     
  41. leandroong

    leandroong Addicted to LI Member

    @Almaz, how do i run your modified script using lighttpd? I initially plan top open it using http://10.0.1.1:81/adblock/almaz.sh but it did'nt work, i'm getting file download option instead.
    note: i'm using padavan FW.
     
    Last edited: Sep 5, 2014
  42. Almaz

    Almaz Serious Server Member

    You don't really need lighttpd. You can put everything in your router folder /tmp/var/wwwext and start it with http://router IP/ext/almaz.sh Make sure to chmod +x almaz.sh before running the script and modify the first lines of configuration. I don't use lighttpd but same applies to lighttpd and you need to change lighttpd conf file to execute .sh files.
     
    Last edited: Sep 5, 2014
  43. leandroong

    leandroong Addicted to LI Member

    1. Padavan FW don't have folder /tmp/var/wwwext, only /tmp/var
    2. lighttpd is setup on folder /opt/share/www. I have webui-aria2 running by calling from url: http://10.0.1.1:81/aria2/index.html. What I did, I setup folder, /opt/share/www/adblock and put all files there. Planning to invoke it using irl: http://10.0.1.1:81/adblock/almaz.sh. Result is download menu.
     

    Attached Files:

  44. Almaz

    Almaz Serious Server Member

    In that case configure the script like that and change your pixelserve ip

    Code:
    #!/bin/sh
    adblockpath="/opt/share/www/adblock/adblock.sh"
    pixelservip="192.168.3.254"      ---> change your pixelservip server IP
    scriptname="almaz.sh"
    dnsmasqlog="/tmp/var/log/messages*"
    ######################################################
     
  45. Almaz

    Almaz Serious Server Member

    Also I forgot, you need to change configuration in lighttpd. From other thread which is not mine.


    See this thread:

    http://redmine.lighttpd.net/boards/2/topics/1580

    That was about getting CGI working with Perl. You'll need those other lines for the alias and the $HTTP["url"], I think.

    I just tried adding ".sh" to the cgi.assign in my lighttpd.conf, so I ended up with:

    cgi.assign = ( ".pl" => "/opt/perl/bin/perl", ".cgi" => "/opt/perl/bin/perl", ".sh" => "/bin/sh" )

    and, after bouncing lighttpd, a test .sh file (foo.sh) works.
     
  46. leandroong

    leandroong Addicted to LI Member

    this is my setting and still fail after changing pixelservip
    adblockpath="/opt/share/www/adblock/adblock.sh"
    pixelservip="10.0.2.254"
    scriptname="/opt/share/www/adblock/almaz.sh"
    dnsmasqlog="/opt/share/www/adblock/log/messages*"

    note: normally needed an extension of .html to work. Maybe, will not work on padavan FW, lacking CGI
     
  47. Almaz

    Almaz Serious Server Member


    It must be
    scriptname="almaz.sh"
     
  48. Almaz

    Almaz Serious Server Member

  49. leandroong

    leandroong Addicted to LI Member

  50. HunterZ

    HunterZ LI Guru Member

    Lighttpd probably doesn't know to execute shell scripts as web server scripts. Almaz's post explains how to fix that.
     
  51. leandroong

    leandroong Addicted to LI Member

    I know, i'm figuring what optware to install for CGI to work and how to configure it. Any idea or steps?
     
  52. HunterZ

    HunterZ LI Guru Member

    No idea here. I didn't look into lighhttpd after people informed me of the user path supported by Tomato's built in web server.
     
  53. leandroong

    leandroong Addicted to LI Member

  54. leandroong

    leandroong Addicted to LI Member

    @HunterZ, ok, I was able to make CGI works on lighttpd. This is what I did
    1. recompile lighttpd from source using tomatoware, tried to make it static it fails. Anyways it works
    2. perform strip to reduce size
    3. replace entware lighttpd with tomatoware compiled version.
    4. create lighttpd adblock folder, /opt/share/www/adblock
    5. copy andreDVJ.sh to /opt/share/www/adblock
    6. edit /opt/etc/lighttpd/lighttpd.conf, add the ff at the start
    server.modules = ( "mod_cgi" )
    cgi.assign = ( ".sh" => "/bin/sh" )
    7. start lighttpd
    8. test run, http://10.0.1.1:81/adblock/andreDVJ.sh, it works but empty

    edit2: here is my andreDVJ script
    adblockpath="/media/optware/adblock/adblock.sh" # jerrm's ADBLOCK SCRIPT
    pixelservip="10.0.1.254" # PIXELSERV'S IP ADDRESS
    scriptname="andreDVJ.sh" # THIS SCRIPT
    dnsmasqlog="/media/optware/adblock/logs/dnsmasq.log" # WHERE dnsmasq STORES ITS LOGS
    ######################################################
    if grep -q 'echo $(wc -l < "$blocklist") > /opt/tmp/adscount' $adblockpath
    then
    echo ""
    else
    sed '/elog "$(wc -l < "$blocklist") unique hosts to block"/ a \echo $(wc -l < "$blocklist") > /opt/tmp/adscount' $adblockpath > /opt/tmp/tmp090; mv /opt/tmp/tmp090 $adblockpath
    chmod +x $adblockpath
    ...

    edit3: Tested working all buttons.
     

    Attached Files:

    Last edited: Sep 6, 2014
  55. Almaz

    Almaz Serious Server Member

    Great job leandroong
     
  56. Almaz

    Almaz Serious Server Member

    On the other hand you could just use Lance nginx static file. Also using strip is only good for jffs to reduce storage file. Whenever you are using strip it takes more RAM and resources because strip works similar to zipping a file. Everytime a file gets accessed/executed it needs to unstrip itself then execute.
     
  57. leandroong

    leandroong Addicted to LI Member

    ok. can you explain what this line is doing ?
    if grep -q 'echo $(wc -l < "$blocklist") > /opt/tmp/adscount' $adblockpath

    I'm curious if $blocklist is correct, it is not constant
     
  58. Almaz

    Almaz Serious Server Member

    It adds an additional line to adblock.sh. It counts the hosts lines and gives you an output how many domains you are blocking.
     
  59. HunterZ

    HunterZ LI Guru Member

    I'm pretty sure that strip just removes all debugging data from the executable, which has no effect on performance: http://serverfault.com/questions/196764/what-is-the-point-of-stripping-a-binary-elf-program
     
  60. leandroong

    leandroong Addicted to LI Member

    Last edited: Sep 6, 2014
  61. HunterZ

    HunterZ LI Guru Member

    Did you configure dnsmasq to log queries to the file that the web script is looking for?
     
  62. leandroong

    leandroong Addicted to LI Member

    I don't know how to do it.

    my setting
    adblockpath="/media/optware/adblock/adblock.sh" # jerrm's ADBLOCK SCRIPT
    pixelservip="10.0.1.254" # PIXELSERV'S IP ADDRESS
    scriptname="andreDVJ.sh" # THIS SCRIPT
    dnsmasqlog="/media/optware/adblock/logs/dnsmasq.log" # WHERE dnsmasq STORES ITS LOGS
    ######################################################

    edit3: dnsmasq.custom content as follow
    conf-file=/media/optware/adblock/blocklist
     
    Last edited: Sep 6, 2014
  63. HunterZ

    HunterZ LI Guru Member

    You need this added to dnsmasq.custom:
    log-queries
    log-facility=/media/optware/adblock/logs/dnsmasq.log
     
  64. leandroong

    leandroong Addicted to LI Member

    Now, I'm getting data on dnsmasq.log, partial data as follows:
    Sep 7 01:14:52 dnsmasq[1962]: started, version 2.68 cachesize 1000
    Sep 7 01:14:52 dnsmasq[1962]: compile time options: IPv6 GNU-getopt no-RTC no-DBus no-i18n no-IDN DHCP DHCPv6 no-scripts TFTP no-conntrack no-ipset no-auth
    Sep 7 01:14:52 dnsmasq-dhcp[1962]: DHCP, IP range 10.0.1.100 -- 10.0.1.130, lease time 1d
    Sep 7 01:14:52 dnsmasq-dhcp[1962]: DHCP, sockets bound exclusively to interface br0
    Sep 7 01:14:52 dnsmasq[1962]: read /etc/hosts - 10 addresses
    Sep 7 01:14:52 dnsmasq[1962]: read /etc/storage/dnsmasq/hosts - 0 addresses
    Sep 7 01:14:52 dnsmasq-dhcp[1962]: read /etc/ethers - 7 addresses
    Sep 7 01:14:52 dnsmasq[1962]: using nameserver 121.1.3.89#53
    Sep 7 01:14:52 dnsmasq[1962]: using nameserver 218.102.23.228#53
    Sep 7 01:14:56 dnsmasq[1962]: query[A] pr.comet.yahoo.com from 10.0.1.2
    Sep 7 01:14:56 dnsmasq[1962]: forwarded pr.comet.yahoo.com to 218.102.23.228
    Sep 7 01:14:56 dnsmasq[1962]: forwarded pr.comet.yahoo.com to 121.1.3.89
    Sep 7 01:14:56 dnsmasq[1962]: reply sg3-comet.yahoo.pr.g01.yahoodns.net is 106.10.200.161
    Sep 7 01:15:00 dnsmasq[1962]: query[A] pr.comet.yahoo.com from 10.0.1.2
    Sep 7 01:15:00 dnsmasq[1962]: forwarded pr.comet.yahoo.com to 121.1.3.89
    Sep 7 01:15:00 dnsmasq[1962]: reply sg3-comet.yahoo.pr.g01.yahoodns.net is 106.10.200.161
    Sep 7 01:15:12 dnsmasq[1962]: query[A] ad-clix.com from 127.0.0.1
    Sep 7 01:15:12 dnsmasq[1962]: forwarded ad-clix.com to 121.1.3.89
    Sep 7 01:15:12 dnsmasq[1962]: reply ad-clix.com is 64.29.145.9
    Sep 7 01:15:12 dnsmasq[1962]: query[AAAA] ad-clix.com from 127.0.0.1
    Sep 7 01:15:12 dnsmasq[1962]: forwarded ad-clix.com to 121.1.3.89

    but no block domain names yet
     
  65. HunterZ

    HunterZ LI Guru Member

    Can you post a few lines from /media/optware/adblock/blocklist ?
     
  66. leandroong

    leandroong Addicted to LI Member

    address=/0.datacollector.coin.scribol.com/10.0.1.254
    address=/0.r.msn.com/10.0.1.254
    address=/005.free-counter.co.uk/10.0.1.254
    address=/006.free-counter.co.uk/10.0.1.254
    address=/007.free-counter.co.uk/10.0.1.254
    address=/008.free-counter.co.uk/10.0.1.254
    address=/008.free-counters.co.uk/10.0.1.254
    address=/00fun.com/10.0.1.254
    address=/011707160008.c.mystat-in.net/10.0.1.254
    address=/0427d7.se/10.0.1.254
    address=/061606084448.c.mystat-in.net/10.0.1.254
    address=/064bdf.r.axf8.net/10.0.1.254
    address=/070806142521.c.mystat-in.net/10.0.1.254
    address=/090906042103.c.mystat-in.net/10.0.1.254
    address=/092706152958.c.mystat-in.net/10.0.1.254
    address=/0c9d8370d.se/10.0.1.254
    address=/0d7292.r.axf8.net/10.0.1.254
    address=/0f36f3.r.axf8.net/10.0.1.254
    address=/0iecfobt.com/10.0.1.254
    address=/0koryu0.easter.ne.jp/10.0.1.254
     
  67. HunterZ

    HunterZ LI Guru Member

    What happens if you try to ping 00fun.com on a client machine, or open it in your browser?
     
  68. leandroong

    leandroong Addicted to LI Member

    C:\Users\lean>ping 00fun.com

    Pinging 00fun.com [74.53.201.226] with 32 bytes of data:
    Request timed out.
    Request timed out.
    Request timed out.
    Request timed out.

    Ping statistics for 74.53.201.226:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

    C:\Users\lean>
    ++++
     

    Attached Files:

  69. HunterZ

    HunterZ LI Guru Member

    Can you run ipconfig /all in a command prompt on the client and post the output?
     
  70. leandroong

    leandroong Addicted to LI Member

    C:\Users\lean>ipconfig /all

    Windows IP Configuration

    Host Name . . . . . . . . . . . . : HPG71-LEANWIN7
    Primary Dns Suffix . . . . . . . :
    Node Type . . . . . . . . . . . . : Mixed
    IP Routing Enabled. . . . . . . . : No
    WINS Proxy Enabled. . . . . . . . : No
    DNS Suffix Search List. . . . . . : BTRouter0

    Wireless LAN adapter Wireless Network Connection 3:

    Connection-specific DNS Suffix . : BTRouter0
    Description . . . . . . . . . . . : Intel(R) Centrino(R) Wireless-N 1000
    Physical Address. . . . . . . . . : 00-26-C7-1E-80-FE
    DHCP Enabled. . . . . . . . . . . : Yes
    Autoconfiguration Enabled . . . . : Yes
    Link-local IPv6 Address . . . . . : fe80::f895:7ea7:5f9c:8826%54(Preferred)
    IPv4 Address. . . . . . . . . . . : 10.0.1.2(Preferred)
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Lease Obtained. . . . . . . . . . : September 06, 2014 10:25:11 PM
    Lease Expires . . . . . . . . . . : September 08, 2014 12:29:42 AM
    Default Gateway . . . . . . . . . : 10.0.1.1
    DHCP Server . . . . . . . . . . . : 10.0.1.1
    DHCPv6 IAID . . . . . . . . . . . : 218113735
    DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-17-84-60-00-C8-0A-A9-4B-C7-F8

    DNS Servers . . . . . . . . . . . : 10.0.1.1
    Primary WINS Server . . . . . . . : 10.0.1.1
    NetBIOS over Tcpip. . . . . . . . : Enabled

    Ethernet adapter Wired:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Realtek PCIe FE Family Controller
    Physical Address. . . . . . . . . : C8-0A-A9-4B-C7-F8
    DHCP Enabled. . . . . . . . . . . : Yes
    Autoconfiguration Enabled . . . . : Yes

    Ethernet adapter Local Area Connection:

    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : VirtualBox Host-Only Ethernet Adapter
    Physical Address. . . . . . . . . : 08-00-27-00-B0-7D
    DHCP Enabled. . . . . . . . . . . : Yes
    Autoconfiguration Enabled . . . . : Yes
    Link-local IPv6 Address . . . . . : fe80::7cfa:7f35:8d2a:28c0%56(Preferred)
    Autoconfiguration IPv4 Address. . : 169.254.40.192(Preferred)
    Subnet Mask . . . . . . . . . . . : 255.255.0.0
    Default Gateway . . . . . . . . . :
    DHCPv6 IAID . . . . . . . . . . . : 923271207
    DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-17-84-60-00-C8-0A-A9-4B-C7-F8

    DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
    fec0:0:0:ffff::2%1
    fec0:0:0:ffff::3%1
    NetBIOS over Tcpip. . . . . . . . : Enabled

    Tunnel adapter Local Area Connection* 11:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
    Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
    DHCP Enabled. . . . . . . . . . . : No
    Autoconfiguration Enabled . . . . : Yes

    Tunnel adapter isatap.BTRouter0:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . : BTRouter0
    Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
    Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
    DHCP Enabled. . . . . . . . . . . : No
    Autoconfiguration Enabled . . . . : Yes

    Tunnel adapter isatap.{3989DD36-09C5-4C29-A911-C6D49D15E7B2}:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Microsoft ISATAP Adapter #4
    Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
    DHCP Enabled. . . . . . . . . . . : No
    Autoconfiguration Enabled . . . . : Yes

    Tunnel adapter isatap.{2C066551-2887-47AA-B557-E4379F8ED122}:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Microsoft ISATAP Adapter #5
    Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
    DHCP Enabled. . . . . . . . . . . : No
    Autoconfiguration Enabled . . . . : Yes

    C:\Users\lean>
     
  71. leandroong

    leandroong Addicted to LI Member

    I'm beginning to wonder that my dns-proxy is not working, cofigure at 127.0.0.1:2053

    edit2: there is a note, can this help?
    ### Tells dnsmasq to forward queries for this domains to DNS 10.25.11.30
    #server=/mit.ru/izmuroma.ru/10.25.11.30
     
    Last edited: Sep 6, 2014
  72. HunterZ

    HunterZ LI Guru Member

    what does the dnsmasq log show for the 00fun.com query made by the Windows client?
     
  73. leandroong

    leandroong Addicted to LI Member

    Sep 7 02:25:49 dnsmasq[1962]: query[A] 00fun.com from 10.0.1.2
    Sep 7 02:25:49 dnsmasq[1962]: cached 00fun.com is 74.53.201.226
    Sep 7 02:25:50 dnsmasq[1962]: reply a883.g.akamai.net is 210.5.102.43
    Sep 7 02:25:50 dnsmasq[1962]: reply a883.g.akamai.net is 210.5.102.27

    +++
    C:\Users\lean>ping 00fun.com

    Pinging 00fun.com [74.53.201.226] with 32 bytes of data:
    Request timed out.
    Request timed out.
    Request timed out.
    Request timed out.

    Ping statistics for 74.53.201.226:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

    C:\Users\lean>

    edit2: I have firefox addon installed, adblock+. I disable per page concern
     
  74. HunterZ

    HunterZ LI Guru Member

    Looks like your dnsmasq is not using the data from the blocklist for some reason.
     
  75. leandroong

    leandroong Addicted to LI Member

    I agree.

    edit2:
    /opt/home/admin # dnsmasq -v
    Dnsmasq version 2.68 Copyright (c) 2000-2013 Simon Kelley
    Compile time options: IPv6 GNU-getopt no-RTC no-DBus no-i18n no-IDN DHCP DHCPv6 no-scripts TFTP no-conntrack no-ipset no-auth

    This software comes with ABSOLUTELY NO WARRANTY.
    Dnsmasq is free software, and you are welcome to redistribute it
    under the terms of the GNU General Public License, version 2 or 3.
    /opt/home/admin #

    edit3:
    /opt/home/admin # iptables -L -n
    Chain INPUT (policy ACCEPT)
    target prot opt source destination
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.254 tcp dpt:80
    DROP all -- 0.0.0.0/0 10.0.1.254
    maclist all -- 0.0.0.0/0 0.0.0.0/0
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
    DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
    ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:51415
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:51415
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6800
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6800
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6801
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6801
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6802
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6802
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6804
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6804
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6805
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6805
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6803
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6803
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6806
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6806
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6807
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6807
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6808
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6808
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6809
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6809
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.1 tcp dpt:6810
    ACCEPT udp -- 0.0.0.0/0 10.0.1.1 udp dpt:6810
    ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp !type 8
    DROP all -- 0.0.0.0/0 0.0.0.0/0

    Chain FORWARD (policy ACCEPT)
    target prot opt source destination
    maclist all -- 0.0.0.0/0 0.0.0.0/0
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
    DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
    ACCEPT tcp -- 0.0.0.0/0 10.0.1.10 tcp dpt:51420
    ACCEPT udp -- 0.0.0.0/0 10.0.1.10 udp dpt:51420
    UPNP all -- 0.0.0.0/0 0.0.0.0/0
    DROP all -- 0.0.0.0/0 0.0.0.0/0

    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination

    Chain UPNP (1 references)
    target prot opt source destination

    Chain bfplimit (0 references)
    target prot opt source destination
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0

    Chain doslimit (0 references)
    target prot opt source destination
    RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x17/0x02 limit: avg 20/sec burst 30
    DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x17/0x02
    RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x17/0x04 limit: avg 1/sec burst 5
    DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x17/0x04
    RETURN icmp -- 0.0.0.0/0 0.0.0.0/0 icmptype 8 limit: avg 2/sec burst 5
    DROP icmp -- 0.0.0.0/0 0.0.0.0/0 icmptype 8

    Chain logaccept (0 references)
    target prot opt source destination
    LOG all -- 0.0.0.0/0 0.0.0.0/0 state NEW LOG flags 7 level 4 prefix "ACCEPT "
    ACCEPT all -- 0.0.0.0/0 0.0.0.0/0

    Chain logdrop (0 references)
    target prot opt source destination
    LOG all -- 0.0.0.0/0 0.0.0.0/0 state NEW LOG flags 7 level 4 prefix "DROP "
    DROP all -- 0.0.0.0/0 0.0.0.0/0

    Chain maclist (2 references)
    target prot opt source destination
    DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC 98:FE:94:2E:A1:D7
    DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC 04:46:65:D4:E8:3B

    Chain urllist (0 references)
    target prot opt source destination
    /opt/home/admin #

    edit4: Only ads block is on beginning of youtube playing
     
    Last edited: Sep 6, 2014
  76. leandroong

    leandroong Addicted to LI Member

    dnsmasq recorded when start playing youtube, no beginning ads

    Sep 7 02:53:42 dnsmasq[1962]: query[A] r8---sn-2aqu-hoae7.googlevideo.com from 10.0.1.2
    Sep 7 02:53:42 dnsmasq[1962]: forwarded r8---sn-2aqu-hoae7.googlevideo.com to 121.1.3.89
    Sep 7 02:53:42 dnsmasq[1962]: reply r8.sn-2aqu-hoae7.googlevideo.com is 122.2.153.211
    Sep 7 02:53:43 dnsmasq[1962]: query[A] www.google.com from 10.0.1.2
    Sep 7 02:53:43 dnsmasq[1962]: forwarded www.google.com to 121.1.3.89
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.231
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.221
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.237
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.222
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.242
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.226
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.212
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.217
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.251
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.236
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.246
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.227
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.216
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.241
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.232
    Sep 7 02:53:43 dnsmasq[1962]: reply www.google.com is 122.2.152.247
    Sep 7 02:53:44 dnsmasq[1962]: query[A] r8---sn-2aqu-hoae7.googlevideo.com from 10.0.1.2
    Sep 7 02:53:44 dnsmasq[1962]: forwarded r8---sn-2aqu-hoae7.googlevideo.com to 121.1.3.89
    Sep 7 02:53:44 dnsmasq[1962]: reply r8.sn-2aqu-hoae7.googlevideo.com is 122.2.153.211
    Sep 7 02:53:44 dnsmasq[1962]: query[A] pagead2.googlesyndication.com from 10.0.1.2
    Sep 7 02:53:44 dnsmasq[1962]: forwarded pagead2.googlesyndication.com to 121.1.3.89
    Sep 7 02:53:44 dnsmasq[1962]: reply pagead46.l.doubleclick.net is 173.194.127.45
    Sep 7 02:53:44 dnsmasq[1962]: reply pagead46.l.doubleclick.net is 173.194.127.57
    Sep 7 02:53:44 dnsmasq[1962]: reply pagead46.l.doubleclick.net is 173.194.127.58
    Sep 7 02:53:44 dnsmasq[1962]: query[A] csi.gstatic.com from 10.0.1.2
    Sep 7 02:53:44 dnsmasq[1962]: forwarded csi.gstatic.com to 121.1.3.89
    Sep 7 02:53:44 dnsmasq[1962]: reply csi.gstatic.com is 173.194.40.159
    Sep 7 02:53:44 dnsmasq[1962]: reply csi.gstatic.com is 173.194.40.151
    Sep 7 02:53:44 dnsmasq[1962]: reply csi.gstatic.com is 173.194.40.143
    Sep 7 02:53:44 dnsmasq[1962]: reply csi.gstatic.com is 173.194.40.152
    Sep 7 02:53:45 dnsmasq[1962]: query[A] googleads.g.doubleclick.net from 10.0.1.2
    Sep 7 02:53:45 dnsmasq[1962]: forwarded googleads.g.doubleclick.net to 121.1.3.89
    Sep 7 02:53:45 dnsmasq[1962]: reply pagead46.l.doubleclick.net is 173.194.127.58
    Sep 7 02:53:45 dnsmasq[1962]: reply pagead46.l.doubleclick.net is 173.194.127.45
    Sep 7 02:53:45 dnsmasq[1962]: reply pagead46.l.doubleclick.net is 173.194.127.57
    Sep 7 02:53:47 dnsmasq[1962]: query[A] plus.googleapis.com from 10.0.1.2
    Sep 7 02:53:47 dnsmasq[1962]: forwarded plus.googleapis.com to 121.1.3.89
    Sep 7 02:53:47 dnsmasq[1962]: reply googleapis.l.google.com is 74.125.31.95
    Sep 7 02:53:48 dnsmasq[1962]: query[A] ssl.gstatic.com from 10.0.1.2
    Sep 7 02:53:49 dnsmasq[1962]: forwarded ssl.gstatic.com to 121.1.3.89
    Sep 7 02:53:49 dnsmasq[1962]: query[A] gp5.googleusercontent.com from 10.0.1.2
    Sep 7 02:53:49 dnsmasq[1962]: forwarded gp5.googleusercontent.com to 121.1.3.89
    Sep 7 02:53:49 dnsmasq[1962]: reply photos-ugc.l.googleusercontent.com is 173.194.127.108
    Sep 7 02:53:49 dnsmasq[1962]: reply photos-ugc.l.googleusercontent.com is 173.194.127.106
    Sep 7 02:53:49 dnsmasq[1962]: reply photos-ugc.l.googleusercontent.com is 173.194.127.107
    Sep 7 02:53:49 dnsmasq[1962]: reply ssl.gstatic.com is 173.194.127.63
    Sep 7 02:53:49 dnsmasq[1962]: reply ssl.gstatic.com is 173.194.127.56
    Sep 7 02:53:49 dnsmasq[1962]: reply ssl.gstatic.com is 173.194.127.55
    Sep 7 02:53:49 dnsmasq[1962]: reply ssl.gstatic.com is 173.194.127.47
    Sep 7 02:53:49 dnsmasq[1962]: query[A] gp4.googleusercontent.com from 10.0.1.2
    Sep 7 02:53:49 dnsmasq[1962]: forwarded gp4.googleusercontent.com to 121.1.3.89
    Sep 7 02:53:49 dnsmasq[1962]: reply photos-ugc.l.googleusercontent.com is 173.194.127.74
    Sep 7 02:53:49 dnsmasq[1962]: reply photos-ugc.l.googleusercontent.com is 173.194.127.76
    Sep 7 02:53:49 dnsmasq[1962]: reply photos-ugc.l.googleusercontent.com is 173.194.127.75
     
  77. leandroong

    leandroong Addicted to LI Member

    with firefox addon disable, I can see a lot of 10.0.1.254
    Sep 7 03:06:26 dnsmasq[1962]: query[PTR] 9.145.29.64.in-addr.arpa from 127.0.0.1
    Sep 7 03:06:26 dnsmasq[1962]: cached 64.29.145.9 is hostedc25.carrierzone.com
    Sep 7 03:06:26 dnsmasq[1962]: query[PTR] 254.1.0.10.in-addr.arpa from 127.0.0.1
    Sep 7 03:06:26 dnsmasq[1962]: config 10.0.1.254 is NXDOMAIN-IPv4
    Sep 7 03:06:26 dnsmasq[1962]: query[PTR] 254.1.0.10.in-addr.arpa from 127.0.0.1
    Sep 7 03:06:26 dnsmasq[1962]: config 10.0.1.254 is NXDOMAIN-IPv4
    Sep 7 03:06:32 dnsmasq[1962]: query[A] weather.service.msn.com from 10.0.1.2
    Sep 7 03:06:32 dnsmasq[1962]: forwarded weather.service.msn.com to 121.1.3.89
    Sep 7 03:06:32 dnsmasq[1962]: query[A] wpad.BTRouter0 from 10.0.1.2
    Sep 7 03:06:32 dnsmasq[1962]: forwarded wpad.BTRouter0 to 121.1.3.89
    Sep 7 03:06:32 dnsmasq[1962]: reply a1859.g2.akamai.net is 124.106.175.49
    Sep 7 03:06:32 dnsmasq[1962]: reply a1859.g2.akamai.net is 124.106.175.10
     
  78. jerrm

    jerrm Network Guru Member

    Post dnsmasq.conf and dnsmasq.custom as they exist on disk, not in the gui config.

    \
     
  79. Almaz

    Almaz Serious Server Member

    Before you start troubleshooting, I suggest to reboot your router and PC.

    Just to let you know from your log file, everything looks good and adblock is working. I didn't check your iptables though. The reason it doesn't block ads because of firefox cache. Once you clear the cache then adblock should start working.

    To make it easier just ssh to your router and run these command and post the output.

    Code:
    find / -name "dnsmasq.conf" -exec cat {} \;
    find / -name "dnsmasq.custom" -exec cat {} \;
    Now open in Windows command prompt and run the following command. Post the output

    Code:
    ipconfig /flushdns
    nslookup 0.datacollector.coin.scribol.com
    EDIT: I just checked your iptables and it is missing Chain adblk.fw. The reason it didn't apply adblock iptables because in the script it's using Tomato variables which probably are not available in your firmware.

    That's how your iptables should look like with adblock

    Code:
    Chain INPUT (policy DROP)
    target     prot opt source               destination
    adblk.fw   all  --  0.0.0.0/0            10.0.1.254
    
    Chain adblk.fw (1 references)
    target     prot opt source               destination
    ACCEPT     tcp  --  0.0.0.0/0            10.0.1.254       tcp dpt:80
    ACCEPT     icmp --  0.0.0.0/0            10.0.1.254       icmp type 8
    REJECT     tcp  --  0.0.0.0/0            10.0.1.254       reject-with tcp-reset
    REJECT     all  --  0.0.0.0/0            10.0.1.254       reject-with icmp-host-prohibited
    DROP       all  --  0.0.0.0/0            10.0.1.254
    
    Chain PREROUTING (policy ACCEPT)
    target     prot opt source               destination
    adblk.fw   tcp  --  0.0.0.0/0            10.0.1.254       tcp dpt:443
    
    Chain adblk.fw (1 references)
    target     prot opt source               destination
    DNAT       tcp  --  0.0.0.0/0            10.0.1.254       tcp dpt:443 to:10.0.1.254:80
    
    
    
     
    Last edited: Sep 7, 2014
  80. HunterZ

    HunterZ LI Guru Member

    No reason to dnat SSL to port 80 if using modern versions of pixelserv, which can listen on multiple ports.

    Also, I disagree that logs show adblock working, as dnsmasq clearly returned a cached non-pixelserv IP for a domain in the blocklist, and I don't see any evidence of it returning the pixelserv IP for any domains. The only mention of the pixelserv IP in the logs is some weird *reverse* lookup.
     
  81. Almaz

    Almaz Serious Server Member


    I already edited my post and confirmed why it doesn't work. He is using a different firmware and script didn't run properly. Iptables are not setup and probably dnsmasq wasn't set as well. By the way should I just put pixelserv in startup script for 443? I'm using a little older version of pixelserv v32 with Jerrm script. For now it's just forwards requests from 443 to 80. Any advantages running a second pixelserv on SSL?
     
    Last edited: Sep 7, 2014
  82. jerrm

    jerrm Network Guru Member

    The missing iptables rules would not prevent adblock or pixelserv from working. They are not the problem.

    When it comes down to it, pixelserv itself is not needed. If dnsmasq and the client's dns are correctly configured, adds will be blocked regardless.

    @leandroong's problem is with dnsmasq, not iptables or pixelserv. Hard for us to debug as he is running a tomato customized script on non-tomato firmware.

    As HZ said, you don't need the dnat rule with a current pixelserv as it will listen on both 80 and 443. You don't need a second pixelserv instance, just a single instance of a current version.
     
  83. Almaz

    Almaz Serious Server Member

    Just want to confirm if I use the latest pixelserv then I can edit your script to delete dnat?
     
  84. leandroong

    leandroong Addicted to LI Member

    Here is the requested files:
    edit: I think, possible problem is lighttpd missing modules. I'm checking it
     

    Attached Files:

  85. jerrm

    jerrm Network Guru Member

    If your using a current version you won't have to. No promises if you've modified it.
     
  86. Almaz

    Almaz Serious Server Member


    Thanks, I'll update pixelserv. So far your script worked great since the beginning.
     
  87. leandroong

    leandroong Addicted to LI Member

    In CGI script, I don't see file adscount being created
    #!/bin/sh
    adblockpath="/media/optware/adblock/adblock.sh" # jerrm's ADBLOCK SCRIPT
    pixelservip="10.0.1.254" # PIXELSERV'S IP ADDRESS
    scriptname="andreDVJ.sh" # THIS SCRIPT
    dnsmasqlog="/media/optware/adblock/dnsmasq.log" # WHERE dnsmasq STORES ITS LOGS
    ######################################################
    cd /media/optware/adblock

    if grep -q 'echo $(wc -l < "$blocklist") > adscount' $adblockpath
    then
    echo ""
    else
    sed '/elog "$(wc -l < "$blocklist") unique hosts to block"/ a \echo $(wc -l < "$blocklist") > adscount' $adblockpath > tmp090; mv tmp090 $adblockpath
    chmod +x $adblockpath
    fi

    edit: also why $blocklist instead of blocklist
     
  88. jerrm

    jerrm Network Guru Member

    As I suspected, it doesn't look like padavan supports the dnsmasq.custom functionality. dnsmasq.custom is a tomato-ism. It's contents should be appended to dnsmasq.conf by the service restart and are not.
     
  89. leandroong

    leandroong Addicted to LI Member

    just added.
    log-queries
    log-facility=/media/optware/adblock/dnsmasq.log
    conf-file=/media/optware/adblock/blocklist

    result is the same, empty.
     

    Attached Files:

  90. Almaz

    Almaz Serious Server Member


    It's possible you can add contents in your router GUI. Look for something DHCP or DNS.
     
  91. Almaz

    Almaz Serious Server Member

    It must be $blocklist because it's a variable in jerrm's adblock.sh.

    Change the code to
    Code:
    if grep -q 'echo $(wc -l < "$blocklist") > /tmp/adscount' $adblockpath
    then
    echo ""
    else
    sed '/elog "$(wc -l < "$blocklist") unique hosts to block"/ a \echo $(wc -l < "$blocklist") > /media/optware/adblock/adscount' $adblockpath > /media/optware/adblock/tmp090; mv /media/optware/adblock/tmp090 $adblockpath
    Also change the following in andreDVJ.sh

    FROM:
    Code:
    echo '<br>hosts'
      if [ -f /tmp/adscount ];
      then
      cat /tmp/adscount
      else
      echo "!!!Restart!!!"
      fi
    TO
    Code:
    echo '<br>hosts'
      if [ -f /media/optware/adblock/adscount ];
      then
      cat /media/optware/adblock/adscount
      else
      echo "!!!Restart!!!"
      fi
     
  92. jerrm

    jerrm Network Guru Member

    Where did you add these? They are not shown in your attachment?

    Be careful adding conf-file directives, dnsmasq will not startup if the file is missing.
     
  93. leandroong

    leandroong Addicted to LI Member

    Now working
     

    Attached Files:

  94. leandroong

    leandroong Addicted to LI Member

    Adblock is working on drama site too.
    Edit: my dnsmasq.log is getting bigger, what maintenance do you recommend?

    edit2: There is no more /etc/dnsmasq.custom file, that is the only changes I notice.
     
  95. HunterZ

    HunterZ LI Guru Member

    There is discussion about dnsmasq log rotation on previous pages. Basically I have a scheduled job that runs once a day to rename the log file, create a new empty one with proper permissions, then poke dnsmasq with SIGUSR2 to tell if to reopen the log file.
     
  96. leandroong

    leandroong Addicted to LI Member

    I create create-dnsmasqlog.sh, will invoke it manually
    #!/bin/sh

    adblockpath="/media/optware/adblock/adblock.sh"
    cd /media/optware/adblock

    #disable adblock
    sh $adblockpath toggle
    killall dnsmasq

    rm /media/optware/adblock/dnsmasq.log
    touch /media/optware/adblock/dnsmasq.log
    sleep 1
    /usr/sbin/dnsmasq
    sh $adblockpath toggle
     
  97. jerrm

    jerrm Network Guru Member

    No need for the toggle in that script.
     
  98. leandroong

    leandroong Addicted to LI Member

    No need to terminate adblock ?

    edit: thanks, I just tested it
     
  99. HunterZ

    HunterZ LI Guru Member

    You don't need to fully kill dnsmasq either, as a USR2 signal will tell it to reopen the log file. I think I posted this earlier, but here is what I do:
    Code:
    # rotate dnsmasq logs
    DNSDIR=/cifs1/adblock
    DNSLOG=${DNSDIR}/dnsmasq.log
    DNSOLD=${DNSDIR}/dnsmasq.old
    echo "Rotating dnsmasq log ${DNSLOG} to ${DNSOLD}..."
    mv -f ${DNSLOG} ${DNSOLD}
    touch ${DNSLOG}
    chmod 777 ${DNSLOG}
    kill -USR2 `cat /var/run/dnsmasq.pid`
     
    pharma and ambiance like this.
  100. leandroong

    leandroong Addicted to LI Member

    very nice, thanks
     

Share This Page