1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Why doesn't Tomato's Access Restriction Block HTTPS Websites?

Discussion in 'Tomato Firmware' started by WRD - EasyTomato, Oct 5, 2012.

  1. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Hi All,

    Does anyone know why Tomato's access restriction doesn't block HTTPS sites? Isn't Tomato using iptables for its access restriction? As both the DNS lookup packet and the HTTPS certificate have the URL (or keyword) in plain text seems like we should be able to find keywords just like we do in HTTP traffic. (at bit more about blocking https using iptables here)

    Also, in regards to this question, OpenDNS isn't an answer for us and some users getting around Tomato's access restrictions isn't a problem for us either.

    Thanks!
    wrd
     
  2. koitsu

    koitsu Network Guru Member

    Because Access Restriction, for blocking HTTP traffic (e.g. a website name), has to look at the packet contents, more specifically, HTTP headers. Specifically for HTTP website names/content, there's a custom kernel module which is used to examine the TCP payload and block the connection being made through the router (i.e. it never makes it out the WAN port). This is what's referred to as "layer 7 filtering" (when something filters based off of packet contents). Access Restriction does use iptables, but it doesn't use deny/allow rules like a classic firewall -- it uses custom iptables modules to do the filtering (re: packet content inspection).

    HTTPS is encrypted from the moment the TCP connection is established. Meaningfrom the very moment the TCP connection is established, all packet contents are encrypted with SSL -- that includes HTTP headers. The entire point to SSL is to keep people from eavesdropping; a router filtering traffic based on packet contents is considered a form of eavesdropping. This is why the encryption starts IMMEDIATELY once the TCP connection is established; all payload from that point forward is encrypted, thus the router cannot see HTTP headers or GET/HEAD/POST requests.

    You only have one choice for filtering HTTPS traffic: use proxying software combined with a firewall rule that either blocks outbound connections to TCP port 443 (or for HTTPS sites which listen on port numbers other than 443, you have to use a layer 7 filter (very very CPU intensive) to determine what is and what is not SSL), in addition to requiring every machine on your LAN have their web browsers/programs/etc. set up to use the proxy server.

    HTTPS proxying is done via HTTP 1.1 using a special method per RFC2616. Instead of a TCP connection being made directly to a site and GET/HEAD/POST commands being issued along with HTTP headers, the client instead issues a TCP connection to the proxy and issues a CONNECT command. In the CONNECT command is the site name being accessed as well as the port number (i.e. "CONNECT www.google.com:443 HTTP/1.1") followed immediately by client headers (e.g. "Host: www.google.com"). The proxy then initiates the TCP connection to www.google.com:443 and acts as a "middle man". This is referred to as "HTTP tunnelling". This can also be used to tunnel other protocols (vs. just HTTPS/SSL) too. Please refer to Wikipedia, and some other resources:

    http://muffin.doit.org/docs/rfc/tunneling_ssl.html
    http://serverfault.com/questions/276467/how-do-proxy-servers-filter-https-websites
    http://stackoverflow.com/questions/516323/https-connections-over-proxy-servers
    http://stackoverflow.com/questions/...o-a-forward-http-proxy-over-an-ssl-connection

    If you familiarise yourself with HTTP (the protocol) all of this will make more sense. And anyone who is doing firewalling or maintaining access restrictions, in my opinion, should definitely take the time to understand the network protocols they're working with. Failure to learn about those protocols makes the person, in my opinion, equivalent to a fish out of water. People need to learn about the underlying technology they're working with, all the way down to the protocol. I cannot stress this enough.
     
  3. lancethepants

    lancethepants Network Guru Member

    Lately I've been wondering if this also might be possible with the latest dnscrypt version 1.1. V1.1 has plugin support that could possibly be used to prevent domains like facebook.com from even being resolved. This happens much before any TCP connection to facebook.com, or any other desired domain, would be established.

    Plugin support allows you to observe the DNS queries before they've been sent. It also allows you to evaluate the response too, before passing it on to the device that made the query. At either point you can decide what to do with the query. You could modify it, send someone to a captive portal, or even just throw it away.

    There are some plugins pre-packaged I've read, but haven't looked to heavily into their usage. I've been wanting to test this out, but haven't yet had the time. Of course there's almost always a way to bypass these things, but this could possibly be a cleaner and more complete way to somewhat cover the existing holes in the system.

    Using this, also with the "Intercept DNS port (UDP 53)" Setting found in the advanced DNS page in tomato, would ensure all queries are forced through DNSCrypt.

    edit: As OpenDNS is the only place that DNSCrypt is supported, this may not be viable. The actual filtering would take place on the router, but DNS queries would be forced to OpenDNS. ie, They would not have to Open an OpenDNS account for this to work. So maybe still a possible solution? Would be nice to see them de-couple those two functionalities (encryption, and filtration). But I think they point is the want people to use their DNS service.
     
  4. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    @koitsu - Thanks for taking the time to write back, but I think you missed the point of the question. The question is can we stop HTTPS websites from loading. Yes, "HTTPS is encrypted from the moment the TCP connection is established." That's true, but I'm not usre we need to read TCP packets nor GET/HEAD/POST requests in order to stop the site from loading.

    I did a wireshark capture on loading twitter.com (which is https). I can see, in plain text, a DNS request go out for twitter.com as well as a Client Hello and Server Hello TLS packets which both contain the string twitter.com. Can't we drop any of these?

    Also, when I put the following line in iptables on an Ubuntu VM, twitter.com will NOT load, but other HTTPS sites are unaffected. So iptables can do selective blocking of https sites (also tried with bankofamerica.com) . (this is from the post about blocking https traffic with iptables)
    Code:
    iptables -t filter -I INPUT -m string --string twitter -j REJECT --algo bm
    I'll be the first to admit I don't know a much about this (not really a CS guy), so I may have missed something, but seems to me, we should be able to make this work.

    @lancethepants - Is viewing DNS queries that hard? I have no idea at all, today is my first day looking at any of that stuff after someone said EasyTomato couldn't block HTTPS traffic (which it cant!)

    Unfortunately having people sign up for OpenDNS accounts is really out of the question for what we're trying to do at EasyTomato. We are fine with a few people being able to bypass this stuff. If we can cover 85% of the users, that's awesome.

    If you end up wanting to mess with this stuff let us know! We'd love to be able to block https too as more and more sites are moving to https by default.
     
  5. lefty

    lefty Networkin' Nut Member

    I have often pondered, though i cannot say i have tried it myself but - most, if not all https sites resolve to the same IP as the http site, if you wanted to block this site, could you not simply add a drop rule with the IP in question to your iptables?
     
  6. koitsu

    koitsu Network Guru Member

    That iptables entry you've used is causing DNS to be blocked, which in turn is causing "the site not to load". Standard (non-DNSSEC) DNS is plaintext, so that's the type of packet that is being matched. DNS is not HTTPS. I can assure you the strings module for iptables cannot decode HTTPS. You asked how to block HTTPS, not how to block DNS -- see subject of thread.

    You're welcome to do DNS-based blocking if you want, but nothing stops a user from editing the hosts file on their local workstation, pointing the site they want to reach to an IP address of that server, and then using https://sitetheywanttoreach/ and bypassing use of the DNS server entirely.

    You're welcome to block TCP port 443 (commonly used for SSL), but if the site operator runs their HTTPS instance on a port different than 443, then your block won't apply. You would have to, in effect, micro-manage your LAN users (standing over them, sniffing their traffic, etc.) to figure out what they were up to.

    Then there are the users who will use an SSH tunnel to a machine somewhere on the Internet to work around all the rest of the limits/etc. you put in place, but maybe that's not a concern on your network. You could then block outbound TCP port 22, but then then the user might ask the server admin to make sshd listen on port 1234, then the next day they're using that. (Yes, this is exactly what happens -- people do it to avoid firewalls).

    If you really want to secure your network and only allow very specific protocols through, you need to change the default model of traffic from allow outbound to any to deny outbound and then pass only what you consider legitimate -- this needs to be done on both the WAN (e.g. WAN->Internet) as well as LAN (e.g. LAN->router). The sets of rules will differ.

    You should talk to your network administrator about these things, I'm sure they can shed light on the subject to the same degree I am.
     
  7. koitsu

    koitsu Network Guru Member

    This won't work with sites/destinations that use anycast, nor will it work with sites that are multi-homed or use RR DNS for A records.

    Let's also not forget that the server/site maintainer could deploy multihoming or anycast at any time, or could simply change hosting providers / change IPs, so suddenly your firewall (iptables) which is blocking a single IP only work "some" of the time. Basically this requires the person maintaining the firewall to constantly try to "micro-manage" the Internet and do CONSTANT research (maybe every hour, every day, etc.) and review of every blocked site to see if there's a change. It's completely 100% unmanageable. The proper way to solve this is to invert the firewall model -- instead of "allow by default, deny selectively", you change it to "deny by default, allow selectively".
     
  8. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    @koitsu - I explicitly said a few times we did not care if some users were able to get around it. We are not trying to secure our network, only make something that will block the average user.

    I didn't ask "how to block HTTPS." The subject of the thread is "Why doesn't Tomato's Access Restriction block HTTPS websites." The question was how to block an HTTPS website from loading, not how to selectively block HTTPS traffic.

    I haven't really looked into it, but as setting up the TLS connection gives the website name in plain text, we could potentially block that. That wouldn't depend on DNS resolution or lack there of (editing hosts file).
     
  9. koitsu

    koitsu Network Guru Member

    Thank you for clarifying.

    I don't know how many times I'm going to have to say this: with HTTPS, the instant the TCP connection is established, all I/O is encrypted. There is nothing submit across the socket in plaintext; it's encrypted from the very first byte onward. There is absolutely nothing in plaintext you can filter off of using the iptables string module, or any other layer 7 filter. Sad but true. Your iptables example earlier worked because it blocked DNS resolution (DNS record lookups are done in plaintext, unless DNSSEC is in use).

    The only SSL-based plaintext "thing" you could filter would be sites using RFC 2817. That RFC defines how to upgrade an already-established plaintext HTTP session to an SSL session. However, despite Apache 2.x supporting this, I have never seen any sites using this. The standard model for HTTPS sites used by 99% of the Internet is to have a listening socket on TCP port 443 that is encrypted from the moment the TCP connection is established. The same model applies to things like POP+SSL and IMAP+SSL. The "TLS upgrade" model is uncommon these days because of exactly that -- there is some plaintext data transmit/received before SSL takes over, thus it is considered insecure and is avoided by most.

    So your choices are:

    1. Block DNS lookups (basically what lancethepants was discussing and what your iptables filter above did),

    2. Force use of an HTTP proxy that permits HTTP CONNECT, since the site name is provided in plaintext during the CONNECT phase (literally the plaintext data looks like "CONNECT www.sitename.com:443 HTTP/1.1" followed by HTTP client headers -- everything past that point is encrypted). This would require an HTTP proxy daemon running somewhere on the local network, as well as an iptables firewall rule on the router to block outbound connections to TCP port 443 from any LAN IP other than the box/server running the HTTP proxy daemon (otherwise people could simply disable proxy usage in their browser and get around things),
     
  10. lancethepants

    lancethepants Network Guru Member

    There is a DNScrypt plugin that will allow you to create a file of blacklisted domains (including wildcard), and block them. It can even block whole domains if at least one of the ip adresses returned from the query is in the blacklist. I'll play around with it this weekend and will return and report.
     
  11. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Have you tried looking at ClientHello and ServerHello messages for a TLS handshake? To save you time, I'll attach a picture of my packet capture (at bottom). This is modern Chrome connecting to Gmail. In plain English it says accounts.google.com, in both ClientHello and ServerHello. Go try it out on a few of your favorite https sites, it's there on all of the ones I tried: Bank of America, Twitter, Gmail.

    Cool man, thanks! Hopefully we can just mod the regular Tomato Access Restriction method to make this work. We've got someone on the EasyTomato dev team that knows 10,000x what I do and thinks maybe we can make this work. His time is pretty limited though, so I'm doing some homework for him. In any case, let me know what you find!

    ClientHello
    [​IMG]
    ServerHello
    [​IMG]
     
  12. gfunkdave

    gfunkdave LI Guru Member

    I'm not sure I understand the OP's problem. When I type "twitter.com" into the HTTP block box in Access Restrictions, any connection attempts to twitter.com - whether http or https - result in a Connection Reset error. So it seems to be working fine.
     
  13. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Doesn't work at all for me. You tried to go to https://twitter.com?
     
  14. gfunkdave

    gfunkdave LI Guru Member

    Yes. Chrome and Firefox both gave a Connection Reset error. Same if I try for http://twitter.com.

    edit: odd. When I try to do this for Facebook, I get the behavior you mention. http requests are successfully blocked, but not https.
     
  15. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Which version of Tomato are you using? You are just putting twitter.com in the http box? You're sure you don't have some sort of vpn turned on?

    Doesn't work for me with tomato-K26USB-1.28.7500.4MIPSR2Toastman-RT-VPN
     
  16. gfunkdave

    gfunkdave LI Guru Member

    I am using v1.28.7500 MIPSR2Toastman-RT K26 USB VPN
     
  17. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Ok, same same. Doesn't work for me at all.
    http://twitter.com is blocked https://twitter.com is not blocked at all. Same goes for any other https site I try.

    There area number of other posts asking why https sites can't be blocked. Either you're not accessing the https version of the site correctly or you've got a magic router. :) You can try doing an https Google search for twitter and clicking on the first link, that won't load?
     
  18. koitsu

    koitsu Network Guru Member

    Wow, I stand corrected -- thank you for correcting me and educating me.

    The first packet selected is what's important -- what you're seeing is SNI, per RFC6066 / RFC4366 / RFC3546. I wasn't aware of this RFC extension (should give you some idea how long I've been doing this, i.e. back to the 90s when we had no such extension :) ).

    Please note that some browsers/libraries do not support this SSL extension, such as presently-used OpenSSL versions by FreeBSD and some other platforms (varies of course), Python 2.x (still heavily in use), and -- probably the most important of all -- the stock browser that comes with Windows XP: IE6.

    The 2nd packet selected contains a partial plaintext version of the public CA. The field you're seeing is the CN (CommonName). It's very possible that a substring match might not match this given use of wildcard CNs.

    The 2nd packet shown isn't of any relevance here, at least not as far as the iptables strings module is concerned -- it's going to block things as early as the first selected packet you showed.

    But I need you to confirm what the module actually does by putting an Access Rules rule in place and testing this out. A screenshot of the packet capture would be fine. I believe what's going to happen is this:

    1. Client issues TCP SYN to some.ip.address TCP port 443, packet gets forwarded to default gateway (router) across LAN interface
    2. Router firewall examines packet (passes), NATs the packet and sends it out the WAN
    3. Server at some.ip.address TCP port 443 responds with SYN+ACK, makes it back through WAN interface of router, gets NAT'd, and back to client
    4. Client issues TCP ACK to complete handshake, does same thing as what's in #1/#2
    5. Client sends TLSv1 packet containing the string pertaining to SNI, packet gets forwarded to default gateway (router) across LAN interface
    6. Router examines packet -- gets blocked due to the iptables substring match
    7. Router on LAN interface sends back something to the client -- probably some fake TCP payload (I know for HTTP it sends back a fake "Forbidden" page, which contains HTML content), as well as RST or FIN / FIN+ACK? Need to verify
    8. Router on WAN interface sends back something to the server -- possibly TCP RST? Possibly fake TCP FIN + FIN+ACK? Need to verify

    While this works fine for HTTP, I'm concerned about what may happen during #7 and #8 for something like SSL. So, packet captures need to be done to determine the behaviour. Two captures need to be done simultaneously -- one on the LAN interface and one on the WAN interface. On my RT-N16, the LAN interface is br0, and the WAN interface is vlan2. Payload needs to be examined too. So on the router, with an Access Rule already in place for something like a string of "twitter.com"

    * LAN capture: tcpdump -v -p -i br0 -l -n -s 0 -w /tmp/ssl_lan.pcap "tcp port 443"
    * WAN capture: tcpdump -v -p -i vlan2 -l -n -s 0 -w /tmp/ssl_wan.pcap "tcp port 443"

    Then from the client visit https://twitter.com/ and see what happens. Let things sit for maybe 5-10 seconds afterwards (important due to firewall TCP settings), then close the browser window (important since TCP sessions sometimes don't go into FIN_WAIT2 state until this point), Ctrl-C the tcpdumps on the router, and examine the two captures in Wireshark.

    Finally, my other concern is that if you advertise this as "a way to block HTTPS", I can assure you that you'd get problem reports where people say it's not working due to use of certain browsers/libraries. So be very specific how you word this, and be sure to note that it can be circumvented and/or may not work with some SSL implementations/libraries.
     
  19. koitsu

    koitsu Network Guru Member

    Please provide output from iptables --list -v. I'm pretty sure this can be explained. If it's not in the iptables rules explicitly (for port 80), then the actual iptables strings module itself is doing the TCP port check (and one would need to look at the source code for that).
     
  20. mstombs

    mstombs Network Guru Member

    Attempt to block just twitter.com on latest EasyTomato http request results in the following iptables rules

    Code:
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
        0    0 ACCEPT    all  --  br0    br0    0.0.0.0/0            0.0.0.0/0
        0    0 DROP      all  --  *      *      0.0.0.0/0            0.0.0.0/0          state INVALID
        4  240 TCPMSS    tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp flags:0x06/0x02 TCPMSS clamp to PMTU
      51 12696 restrict  all  --  *      vlan2  0.0.0.0/0            0.0.0.0/0
      98 23170 ACCEPT    all  --  *      *      0.0.0.0/0            0.0.0.0/0          state RELATED,ESTABLISHED
        0    0 wanin      all  --  vlan2  *      0.0.0.0/0            0.0.0.0/0
        2  120 wanout    all  --  *      vlan2  0.0.0.0/0            0.0.0.0/0
        2  120 ACCEPT    all  --  br0    *      0.0.0.0/0            0.0.0.0/0
        0    0 upnp      all  --  vlan2  *      0.0.0.0/0            0.0.0.0/0
     
    Chain OUTPUT (policy ACCEPT 76 packets, 14170 bytes)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain restrict (1 references)
    pkts bytes target    prot opt in    out    source              destination
      51 12696 rres01    all  --  *      *      0.0.0.0/0            0.0.0.0/0
     
    Chain rres01 (1 references)
    pkts bytes target    prot opt in    out    source              destination
        0    0 REJECT    tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          web --hore "twitter.com" reject-with tcp-reset
    
    This blocks http, but not https (browser Chrome on Linux host)

    IIRC the web target is Tomato's Jon Zarate custom code, which I would not be surprised only checks http, there may well be better string tools now?
     
  21. koitsu

    koitsu Network Guru Member

    This situation is ugly (as I see it), and very hard to discern what's going on. I have not the slightest idea what original code base "EasyTomato" is based on, but with Toastman builds, the ipt_web module (what mstombs calls "the web target") is no longer used circa December 2010. It's still provided/built on Toastman builds (really! It's /usr/lib/iptables/libipt_web.so) but it's no longer used:

    http://repo.or.cz/w/tomato.git/commit/88ac6317b48c664f324d967736164dbf6c99185e

    The reason for this is almost certainly because of this thread (you have to read the entire thing), where it was discovered Zarate's ipt_web module did not work with Linux 2.6 kernels. It looks to me like ipt_web is actually based on a piece of code written by CyberTAN (the company who Linksys subs their coding work out to -- been this way since the WRT54G days) called ipt_webstr.

    Instead, there is a new module called xt_web which is used, which is part of netfilter. The difference? I have no idea, I'd need to go look at the code.

    If Access Rules are not enabled, you won't find the module loaded when doing "lsmod".

    Backtracking for a moment and focusing on ipt_web: the source is in release/src/router/iptables/extensions/libipt_web.c. Looking at that code, it appears to me that the only pre-requisite it has is use of TCP; it does not appear to limit itself to any specific port, only the TCP protocol (see the parse() function; read through the code slowly). The include file that's relevant to this is release/src/linux/linux/include/linux/netfilter_ipv4/ipt_web.h (wonderful pathnames; sigh, Linux kernel...).

    Thus I'm not sure why the code isn't matching strings in the HTTPS SNI packet. Most probable explanation? My lack of familiarity in the design methods behind netfilter and iptables (I'm talking about the actual API/model). :) I don't see any actual *comparisons* going on within ipt_web.c (meaning between provided strings and actual TCP payload data structures), for example, so netfilter must be doing some magic of some sort itself. But I don't see any explicit "this is specific to TCP port 80" bits. Possibly there are underlying netfilter code pieces that cause that prerequisite to happen rather than ipt_web.c.

    Fast forwarding back to Toastman builds:

    There's a technical write-up on how the string module works, but it doesn't look like the netfilter string module is enabled in the kernel:

    Code:
    root@gw:/tmp/home/root# cat /proc/net/ip_tables_matches
    account
    u32
    icmp
    tcpmss
    state
    multiport
    multiport
    mark
    mac
    limit
    iprange
    iprange
    connmark
    connlimit
    connbytes
    udplite
    udp
    tcp
    
    I urge everyone reading this thread to read the last link very very closely. Do not skim it -- read it. The netfilter authors explicitly state that for HTTP/HTTPS filtering, the job is best done using a proxy. I'm glad to see their opinion mirrors my own. :)
     
  22. lancethepants

    lancethepants Network Guru Member

    So I finally got around to playing with the DNSCrypt plugin functionality. I got it to work as expected. I was able to create a file of blacklisted domains and ips, and have them prevented from being successfully queried.

    Since this is using DNS instead iptables, I did notice one nuance. The iptables method is rather immediate, since it comes into affect as soon as it has been added to the firewall rules set. DNS on certain domains took from 0 to x amount of minutes depending. I think this is because when DNS is queried, it is also accompanied with a TTL (Time to live). This tells the PC how long it should cache the query. During the time the TTL is active, the PC does not actually create an external DNS query, but just continues to use what has been cached. Only once the TTL has run out, does is ask the router again for a DNS lookup. This is really subject to the domain holder to decide what the TTL is, but most that I noticed were within only minutes, so still relatively fast for most sites. I would guess it's smart for most domains to keep a smaller TTL, to ensure a quick IP address switch over in case of emergency or change in their structure. Perhaps another unwritten plugin could override the TTL, to force frequent lookups. (More system intensive, but I imagine far from crippling)

    Once a site's DNS queries were blocked, you average Joe would then not be able to access his favorite time waster over http or https. I blocked google and facebook, and almost immediately received complaints from others that they could not access anything. I forgot I had blocked google, and realized how crippling it was :)

    Yes people may google (or bing as it was in my case) the IP addresses of these sites, but I think you'll easily cover your 85% Now this was all while using DNSCrypt which in turn uses OpenDNS, but you DO NOT have to sign up for an OpenDNS account. You would only need a page to create a list of banned sites.

    Now this plugin, may not be sufficient for your needs as it stands. You guys probably want to block sites depending on how you've classified the computer into groups in your firmware. This plugin however is pretty basic, and just performs the blockage system wide. You would need your programmer (or someone else with the knowledge and willingness), to modify the plugin to be able to intelligently filter depending on who is requesting the lookup. Looks like the DNS queries contain both MAC and IP Adresses, so you could probably filter using one of those. It is using ldns (which you would also need to compile and package into tomato), which I know little of, other than it helps make manipulating DNS queries easier.

    The base infrastructure is all here, and just with some tweaking I think you could have a much more effective in house web blocking system.
     
  23. koitsu

    koitsu Network Guru Member

    @lancethepants -- your understanding of how DNS TTL works is completely 100% correct, including situations where you might set a TTL to something small. There is the TTL in the zone SOA record, and there is per-record TTL as well (e.g. a unique TTL for an A record, NS record, MX record, etc.). If a record lacks an individually-set TTL, then it uses the TTL from the SOA record.

    "Spoofing" a different TTL (when handing back a TTL to a LAN client) is completely doable -- dnsmasq, for example, provides this feature using the --max-ttl option. Note that --local-ttl is not relevant to this situation (these are upstream DNS server lookups not DHCP-registered DNS nor /etc/hosts entries), nor is --neg-ttl (that's for domains/zones which lack a SOA record altogether (I have never seen a zone lack a SOA record before).

    Be aware that setting --max-ttl too low (say, 1 second) can greatly hinder the "general Internet experience" for folks, since every single DNS query effectively is cached (locally on the workstation) for 1 second. The workstations then are constantly pounding/banging on their defined DNS server (the router) for DNS queries; DNS is fast but not as fast as something that's locally stored in memory (local resolver/caching nameserver) on the workstation. :)

    And never, ever pick a TTL (in a SOA, per-record, or forced in any way) of 0. This breaks all sorts of things, including enterprise-grade equipment like BlueCoat devices and so on (I dealt with this at my past job; some client of ours had set a per-record TTL on one of their FQDNs to 0, and the BlueCoat device cached the record indefinitely (specifically for 4294967296 (2^10) seconds). That bug BlueCoat's resolver has since been fixed, but if you really need something to "never be cached" or "be cached for as short as possible", 1 second is best.
     
  24. mstombs

    mstombs Network Guru Member

    @koitsu - EasyTomato is based off very recent Toastman RT-N, I expect exactly the same issue with other versions, this area of code hasn't changed for years as far as I can tell

    The actual matching is done in the kernel, you refer to the iptables command processing to insert the rule

    Code:
    /tomato/release/src/linux/linux/net/ipv4/netfilter/ipt_web.c
    This link to the file works for me...
    https://github.com/ReliefLabs/EasyT.../src/linux/linux/net/ipv4/netfilter/ipt_web.c

    There's another similar function ipt_webstr.c in same folder which is not compiled AFAIK.

    Do https requests start with "GET" or "POST"?
     
  25. koitsu

    koitsu Network Guru Member

    @mstombs, please look at this commit, and its date. Read the commit comment:

    http://repo.or.cz/w/tomato.git/commit/88ac6317b48c664f324d967736164dbf6c99185e

    Or more specifically, this:

    http://repo.or.cz/w/tomato.git?a=search&h=refs/tags/Toastman-1.28.0500.5&st=commit&s=ipt_web

    These results are for Toastman's 1.28.0.500.5 build. Note the commit dates.

    So are you absolutely 100% certain that ipt_web is still used? The commit there indicates that xt_web is used. Please get on a EasyTomato router and do "lsmod" and provide the output here.

    xt_web module != ipt_web module != strings module.

    You're correct the ipt_webstr module is built but not used/loaded anywhere from within the firmware code. I imagine it's still compiled/built and included because people want to try and minimise the number of differences between that and the Linksys stock firmware (which is where it comes from).

    As for this question:

    The answer is: it doesn't matter. The text filtering cannot see the GET or POST requests -- by that point the certificates have been negotiated and all network I/O is encrypted. The router cannot see what kind of request is being made (GET, POST, HEAD, etc.). You can submit any kind of request (GET, POST, HEAD, etc.) over HTTPS. Remember: HTTPS is just HTTP, with an SSL/TLS and certificate negotiation phase that happens beforehand. Once the negotiation completes, things are 100% encrypted, so anything "in the middle" (like a router) can't see anything plaintext in the TCP payload.
     
  26. mstombs

    mstombs Network Guru Member

    You are correct, xt_web looks to be the ipv4 or v6 universal version, I'm not into ipv6 so maybe the source on my computer was last used for a K24 build? but function looks very similar. I linked to the EasyTomato source since there are so many branches in tomatousb git, but looks like that code wasn't relevant

    Code:
    Tomato v1.28.7500 MIPSR2BETA K26 USB VPN-NOCAT
    root@unknown:/tmp/home/root# lsmod
    Module                  Size  Used by    Tainted: P
    xt_web                  2016  1
    nls_cp437              4416  1
    ip6table_mangle          992  0
    ip6table_filter          704  0
    xt_IMQ                  736  0
    imq                    2320  0
    usblp                  11312  0
    ehci_hcd              34640  0
    vfat                    9216  1
    fat                    46000  1 vfat
    ext2                  55520  0
    ext3                  113440  0
    jbd                    48352  1 ext3
    mbcache                4528  2 ext2,ext3
    usb_storage            33120  1
    sd_mod                21376  2
    scsi_wait_scan          384  0
    scsi_mod              75488  3 usb_storage,sd_mod,scsi_wait_scan
    usbcore              114736  4 usblp,ehci_hcd,usb_storage
    nf_nat_pptp            1440  0
    nf_conntrack_pptp      3808  1 nf_nat_pptp
    nf_nat_proto_gre        1072  1 nf_nat_pptp
    nf_conntrack_proto_gre    2464  1 nf_conntrack_pptp
    nf_nat_ftp              1568  0
    nf_conntrack_ftp        5792  1 nf_nat_ftp
    nf_nat_sip              5920  0
    nf_conntrack_sip      19008  1 nf_nat_sip
    nf_nat_h323            5504  0
    nf_conntrack_h323      37152  1 nf_nat_h323
    wl                  2610544  0
    dnsmq                  2032  1 wl
    et                    37344  0
    igs                    13584  1 wl
    emf                    17568  2 wl,igs
    
    Thinking about it it was only http 1.1 that required the host: field, so the current code might not catch http 1.0?
    Is there any chance to break these https connections on the very first connection where there must be the host line in plain text?
    I guess this check has to be made on all packets, as webservers can re-use established connections, so checking new only not sufficient.
    I imagine it is quite costly in terms of cpu/throughput to check every data packet, so no surprise fastnat skips this module!
     
  27. lancethepants

    lancethepants Network Guru Member

    @koitsu
    Thanks for the deeper insight on DNS and TTL. Looks to me that using the DNSMasq --max-ttl option would be perfect to use in conjunction with what I was experimenting with. I especially like that it only affects the client's TTL, whereas the router still caches the true TTL, and only performs new upstream queries when it runs up.
     
  28. koitsu

    koitsu Network Guru Member

    @mstombs, the Host: header isn't visible by the router or anything else by that point -- everything is encrypted by that point.

    What WRD - EasyTomato and I are discussing has to do with blocking the plaintext strings found in the actual SSL certificates and/or CA. There's a field in the SSL certificate request called SNI (please see my previous posts, I provide a link to what this is) which is in plaintext; this gets submit before the underlying HTTP protocol gets encrypted/encapsulated with SSL, so it's the only thing you can key off of that's in plaintext.

    If a browser or underlying SSL engine provides SNI, a plaintext string-matching module could key off of this and block it. WRD - EasyTomato even confirms that the "strings" module on his Ubuntu box (not the same as xt_web!) works just fine for blocking this kind of thing.
     
  29. mstombs

    mstombs Network Guru Member

    OK, catching on slowly, sorry, ... tomatousb has xt_string in the source, wonder if it can be compiled in and do anything? But maybe perfomance hit would be huge on a router.
     
  30. koitsu

    koitsu Network Guru Member

    Yes, someone would need to build a firmware with that option enabled in the netfilter configuration. How to do that is beyond me -- I haven't done Linux administration + kernel + userland rebuilding since 1997.

    There would be no performance hit unless the module was actually referenced and used within an iptables rule/filter.

    How much of a performance hit is unknown; someone will need to do proper (that means non-half-ass) benchmarks. But based on a quick source code review I can't see it hurting things that much worse than xt_web.
     
  31. mstombs

    mstombs Network Guru Member

    Well the EasyTomato team have done a great job with github source - it just compiles first time, so manually adding xt_string to kernel config_base and iptables makefile does work:-

    Code:
    Tomato v1.28.0000 MIPSR2MST1 K26 USB VPN-NOCAT
    root@unknown:/tmp/home/root# uname -a
    Linux unknown 2.6.22.19 #4 Mon Oct 8 01:58:51 BST 2012 mips GNU/Linux
    root@unknown:/tmp/home/root# insmod xt_string
    root@unknown:/tmp/home/root# lsmod
    Module                  Size  Used by    Tainted: P
    xt_string                864  0
    ...
    STRING match v1.3.8 options:
    --from                       Offset to start searching from
    --to                         Offset to stop searching
    --algo                        Algorithm
    --string [!] string          Match a string in a packet
    --hex-string [!] string      Match a hex string in a packet
    root@unknown:/tmp/home/root# iptables -I FORWARD 1 -m string --string "twitter.com" --algo bm  --from 1 --to 600 -j REJECT
     
    root@unknown:/tmp/home/root# iptables -nvL
    Chain INPUT (policy DROP 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
        0    0 DROP      all  --  *      *      0.0.0.0/0            0.0.0.0/0          state INVALID
    1449  105K ACCEPT    all  --  *      *      0.0.0.0/0            0.0.0.0/0          state RELATED,ESTABLISHED
        0    0 ACCEPT    all  --  lo    *      0.0.0.0/0            0.0.0.0/0
      132  8592 ACCEPT    all  --  br0    *      0.0.0.0/0            0.0.0.0/0
        0    0 ACCEPT    udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp spt:67 dpt:68
     
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
      175  217K REJECT    all  --  *      *      0.0.0.0/0            0.0.0.0/0          STRING match "twitter.com" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    ...
    
    and https:/twitter.com blocked, as per last post of link in first post of this thread
     
  32. koitsu

    koitsu Network Guru Member

    Okay, so two confirmations that xt_string works, with mstombs confirming that xt_string on EasyTomato actually works.

    Can someone with this working on a EasyTomato or Toastman-based router with xt_string loaded and a iptables rule/block in place for "twitter.com" please perform the necessary packet captures I mentioned:

    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-196009

    ...as I really need to see what happens on the packet level to ensure that this isn't going to cause lots of annoyances on the server side.

    You can upload the captures here if you want me to do the analysis, or if you're familiar with Wireshark and extremely familiar with TCP stack behaviour you can do the analysis yourself.

    Be aware that the pcap filter strings I provided will match any TCP port 443 traffic flowing between LAN systems and the router, as well as the router and the Internet. So if you happen to be accessing something over HTTPS that isn't related to twitter, it's going to show up in the packet capture. Be aware of that before uploading the captures, is what I'm saying (not that anyone can glean any personal information from an HTTPS capture -- that's the entire point!).
     
  33. lancethepants

    lancethepants Network Guru Member

    I've setup a server using SNI, as I wanted more than one https site hosted under 1 ip. As I understand, I don't think you can always depend on SNI being enabled over https. SNI is to make sure that the client receives the correct SSL cert when trying to connect a site that is hosted with other https enabled sites at one ip address. If however only a single https site is being hosted at that ip, sni probably isn't needed, as there won't be an issue with giving the client the wrong SSL cert. In that case I think the whole connection will be encrypted, and the string matching method may not work.
     
  34. koitsu

    koitsu Network Guru Member

    @lancethepants -- absolutely correct. I cover that in my long post here (especially in the last paragraph), including some browsers and SSL libraries/stacks which do not have RFC6066 / RFC4366 / RFC3546 support at all.
     
  35. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Doesn't sound like any major issues came to light. A few browsers getting around this is fine. It's going to be a tiny number and people running those browsers will be smart enough to get around this anyway... Also not worried about SNI stuff as few people are going to put a rule to block lacethepants' HTTPS site.:) The major websites that people are going to want to block will all have nice standard certificates/servers.

    Keywords also work better than domains so encouraging people to block by keyword works better (wider net). Biggest issue is if what they think the site is named doesnt match the Cert. Example is gmail, which is applications.google.com or something like that. Can't do much about this though.

    So what's the verdict gents? Can we get this working easily? Who wants to help? ;)
     
  36. koitsu

    koitsu Network Guru Member

    I've asked twice but nobody has stepped up to the plate. I still need to see packet captures:

    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-196009
    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-196337

    Again, let me make this crystal clear: the last thing you want to do is start using something that's going to piss off server administrators. This has already happened in the past with commercial router companies -- some examples:

    http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse#Notable_cases

    Please do not be hasty in implementing this. I have no problem with what's being implemented, it's just that packet analysis needs to be done first to see what the implications are on the server side.

    My opinion as of this writing (i.e. until I get packet captures), is that lancethepants' model/method should be used for this kind of blocking.
     
  37. mstombs

    mstombs Network Guru Member

    I won't be able to get to do tcpdump, especially the wan side of the router for a while.

    I do understand that breaking the comms in this way can have side effects, and likely to be plenty of retries etc.

    In meantime I assume the correct place to insert a string match block rule would be in the current FORWARD chain which is only applied to outgoing comms?

    I wonder why I can view this forum page with the rule in place on all comms via the router, maybe just luck that website doesn't appear in first 600 bytes of 1500 byte tcp packets?

    I do agree dns block would be best, not sure how much you can do with IP address alone - virtually everything you click on wants to use the dns name. IMHO too early to jump on dnscrypt, isn't DNSSEC coming soon, just like IPV6?

    Could put the string block on comms from the router which would stop dnsmasq finding out the multiple ip addresses belonging to sites? How about using a variant of the geoip match target with a regularly updated list of blacklisted IPs?
     
  38. lancethepants

    lancethepants Network Guru Member

    DNSCrypt definitely has the maturity and stability to handle this, and works perfectly when I deploy it. It's the best way I can think of for simple effective in house web blocking. Just a matter of someone programming it for multi-user awareness if its to be included in Easy Tomato.

    Probably the simplest form of DNS blocking is just to manually place some entries in the hosts file. Could be done in a startup script.
    ie
    127.0.0.1 facebook.com
    127.0.0.1 www.facebook.com

    The one thing DNSCrypt has over the hosts file is that it can handle the wildcard characters.
    ie
    facebook.com
    *.facebook.com
    friendface.com
    *.friendface.com
    *.xxx
    *pr0n*
     
  39. mstombs

    mstombs Network Guru Member

    dnsmasq already does domain blocking, which is equivalent to the *. variants, we use that for adblocking, but hosts file still useful for specific blocks, can't easily use IP address as they dynamically change:-

    Code:
    root@unknown:/tmp/home/root# nslookup twitter.com
    Server:    127.0.0.1
    Address 1: 127.0.0.1 localhost
     
    Name:      twitter.com
    Address 1: 199.59.148.82 r-199-59-148-82.twttr.com
    Address 2: 199.59.150.39 r-199-59-150-39.twttr.com
    Address 3: 199.59.150.7 r-199-59-150-7.twttr.com
    root@unknown:/tmp/home/root# nslookup twitter.com
    Server:    127.0.0.1
    Address 1: 127.0.0.1 localhost
     
    Name:      twitter.com
    Address 1: 199.59.150.39 r-199-59-150-39.twttr.com
    Address 2: 199.59.150.7 r-199-59-150-7.twttr.com
    Address 3: 199.59.148.10 r-199-59-148-10.twttr.com
    
    ps had to bypass router block to post this!
     
  40. azdps

    azdps LI Guru Member

    I compiled the latest toastman's tomato firmware from the git. Anyways I'm able to block http and https access from access restriction in the gui if i add 2 entries for the website i want blocked. need one for Src IP address and Dst IP. otherwise the https would not be blocked.
     
  41. koitsu

    koitsu Network Guru Member

    This is too vague and doesn't make a lot of sense (I've sat here for a good 20 minutes trying to figure it out) from a networking level. Can you please provide a screenshot of your Access Rules entry that shows exactly what it is you configured/changed/etc.? Be specific and please do not edit out anything.
     
  42. azdps

    azdps LI Guru Member

    Sorry for being so vague it was not intentional. I was on a time restraint because I had to goto work. Anyways I've compiled toastmans tomato lite build for my RT-N66U. I compiled it without net filter xt_string support and also with it. I did not have a chance to do a tcpdump or use wire shark.

    I can't give you screenshots right now since I'm at work but let me describe my settings as best as possible. Under the tomato GUI access restriction I have one restriction that has two website entries.

    It's enabled
    Both everyday and always selected
    I selected dst ip and put twitter.com and saved it
    I then added src ip and also put twitter.com and saved it
    No other changes
    Rebooted router
    Also had to make sure Firefox was completely closed down as well
    I can't access twitter.com using either http or https.

    When I get home I'll take some screenshots etc and try to do a tcpdump etc
     
  43. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Alright, it works! You put it in right below the Layer 7 box. I hadn't even seen that line before to be honest!

    Below is the relevant iptables output. The DROP's are from the new part which looks to have done a reverse DNS lookup (so we can only block domains with this, not keywords) and the REJECT is from blocking the HTTP request for "twitter" (how we were doing it before).

    This was really helpful! Thanks guys. We'll update EasyTomato to make sure it can do this type of blocking in the next release.

    Code:
    Chain rres00 (1 references)
    target    prot opt source              destination       
    DROP      all  --  anywhere            r-199-59-150-39.twttr.com
    DROP      all  --  anywhere            r-199-59-148-82.twttr.com
    DROP      all  --  anywhere            r-199-59-150-7.twttr.com
    DROP      all  --  r-199-59-150-7.twttr.com  anywhere         
    DROP      all  --  r-199-59-150-39.twttr.com  anywhere         
    DROP      all  --  r-199-59-148-82.twttr.com  anywhere         
    REJECT    tcp  --  anywhere            anywhere            web --hore "twitter" reject-with tcp-reset 
     
  44. mstombs

    mstombs Network Guru Member

    Trouble with using IP address is that the dns look-up is done by iptables when the rule is loaded and it is the numeric ip address used by the kernel netfilter. Twitter, for example, clearly use more than those 3 ip addresses, your list only matches 2 out of 3 of those I get and not the top one

    Code:
    Non-authoritative answer:
    Name:    twitter.com
    Address: 199.59.149.230
    Name:    twitter.com
    Address: 199.59.150.39
    Name:    twitter.com
    Address: 199.59.148.82
    
    I do think you start simply with blocking user dns, tick box to force users to use router dnsmasq and put

    Code:
    address=/twitter.com/192.168.0.1
    in the custom config

    in my case that diverts to my router pixelserv

    You don't get far with twitter just bypassing dns since the web page links all seem to use twitter.com in urls, other sites may work better if all links are relative.

    But I guess running dnscrypt on your PC would bypass router dns ...

    I haven't tried but I don't think iptables uses dnsmasq for dns lookups, the uclibc does its own using resolvers, if it doesn't a combination of these methods could block your own access to the router I guess!
     
  45. koitsu

    koitsu Network Guru Member

    What mstombs said is correct -- that method will not work consistently, and for the exact reason mstombs says. Please do not use this method as it will not work consistently -- it does not do the same thing as what xt_string and ipt_web do (layer 7 filtering / packet content matching). Let me explain what's going on -- again, this is further evidence that people doing this sort of thing need to have familiarity with protocols!

    When you do a DNS A record lookup (for twitter.com or www.twitter.com), their authoritative nameservers return multiple A records. These A records change constantly due to their load balancers (probably DNS-based LBs like Citrix NetScalers). Here is valid 100% proof:

    Code:
    08:49:46 ~ $ dig a twitter.com.
    ...
    ;; QUESTION SECTION:
    ;twitter.com.                   IN      A
    
    ;; ANSWER SECTION:
    twitter.com.            2       IN      A       199.59.148.82
    twitter.com.            2       IN      A       199.59.148.10
    twitter.com.            2       IN      A       199.59.150.39
    ...
     
    08:49:49 ~ $ dig a twitter.com.
    ...
    ;; QUESTION SECTION:
    ;twitter.com.                   IN      A
    
    ;; ANSWER SECTION:
    twitter.com.            17      IN      A       199.59.149.230
    twitter.com.            17      IN      A       199.59.148.82
    twitter.com.            17      IN      A       199.59.150.39
    ...
    
    Note the DNS TTL changing 3 seconds later, and note that the TTL time changes dynamically as well (e.g. it does not go from 2, 1, 30, 29, 28, etc...).

    So to make it clear: the method azdps proposed does not block things using ipt_web nor xt_string. All it does is a simple DNS lookup at that moment in time, and those A records are blocked. The instant someone's local resolver cache expires, twitter.com will then resolve to a new set of IPs, which may or may not contain a different A record that hasn't been blocked.

    Again: this method DOES NOT USE xt_string or ipt_web.
     
  46. koitsu

    koitsu Network Guru Member

    You're correct -- iptables' DNS code uses libc (in this case uClibc) for resolution, which in turn will rely on whatever is in /etc/resolv.conf (for most people this is their ISPs DNS servers, rather than, say, 127.0.0.1 which would refer to dnsmasq). TL;DR -- your understanding is correct. :)
     
  47. mstombs

    mstombs Network Guru Member

    I wouldn't be surprised if twitter.com (and many other similar popular sites) use all of 199.59.148-150.x, which is >750 IP addresses but could be neatly manually blocked. I also expect they have entirely different ranges for global access any hot swapping on DOS attacks etc.
     
  48. koitsu

    koitsu Network Guru Member

    Per ARIN, Twitter owns 199.59.148.0/22, which is 199.59.148.0 through 199.59.151.255 -- that's 256*4, or 1024, IP addresses. However, they almost certainly own other netblocks. Like you said, if they change/expand, suddenly the IP-based filtering model stops working (or works intermittently). So the simple version is that this model doesn't scale for generic end-users, which I think is what EasyTomato is focused on.

    I think sticking to the xt_string method is best, but like I said earlier, I need to see tcpdump packet captures taken for the WAN and LAN interfaces on a router using that module (with a block in place) before I can say "go for it!" My concern, like I mentioned, is that it may cause massive numbers of SSL errors in the logs on the server side, which will then bring negative attention to things. We don't want that happening.
     
  49. azdps

    azdps LI Guru Member

    mstombs can you tell me how you were able to get xt_string included into the firmware and make it work? I used make menuconfig to enable xt_string and I'm able to load it and it shows up using the command lsmod. I would like to know what changes you made to the iptables makefile though. I attempted to pm you but you have a restricted profile. The currently receive the following error attempting to use a string filter:

    iptables v1.3.8: Couldn't load match `string':File not found

    but I believe making the changes you made to the iptables makefile would solve my problem.

    Also I would like to know your thoughts on this custom iptables match module:
    https://forum.openwrt.org/viewtopic.php?id=17249
     
  50. mstombs

    mstombs Network Guru Member

    I do receive pms from some people - but why not discuss in public!

    I have been 'hacking' Tomato for a while, but am still learning - I do know that "make menuconfig" in the kernel folder unlikely to work for long - Tomato Makefile dynamically edits a copy of config_base, and the correct way to add this would be to edit the Makefile, and optionally do this

    Code:
    diff --git a/release/src-rt/linux/linux-2.6/config_base b/release/src-rt/linux/linux-2.6/config_base
    index e6c2db0..19caad3 100644
    --- a/release/src-rt/linux/linux-2.6/config_base
    +++ b/release/src-rt/linux/linux-2.6/config_base
    @@ -427,7 +427,7 @@ CONFIG_NETFILTER_XT_MATCH_LAYER7=m
    # CONFIG_NETFILTER_XT_MATCH_LAYER7_DEBUG is not set
    # CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
    # CONFIG_NETFILTER_XT_MATCH_TIME is not set
    -# CONFIG_NETFILTER_XT_MATCH_STRING is not set
    +CONFIG_NETFILTER_XT_MATCH_STRING=m
    # CONFIG_NETFILTER_XT_MATCH_WEBSTR is not set
    CONFIG_NETFILTER_XT_MATCH_WEB=m
    CONFIG_NETFILTER_XT_MATCH_WEBMON=m
    
    Also the iptables makefile is clearly not 'auto configured', so I just looked for the 'web' target and added string to the list of objects. Probably also had to do "make clean" to force recompilation due to broken dependency checking. Again for size sensitive builds this Makefile shouldn't enable support for things the kernel (or extras) doesn't support.

    Code:
    diff --git a/release/src/router/iptables/extensions/Makefile b/release/src/router/iptables/extensions/Makefile
    index 7ddfd74..c6896d8 100644
    --- a/release/src/router/iptables/extensions/Makefile
    +++ b/release/src/router/iptables/extensions/Makefile
    @@ -13,7 +13,7 @@ PF_EXT_SLIB:=CLASSIFY CONNMARK DNAT DSCP LOG MARK MASQUERADE REDIRECT REJECT
    PF_EXT_SLIB+=ROUTE SNAT TCPMSS TOS TRIGGER TTL NETMAP
    PF_EXT_SLIB+=condition connlimit connmark geoip icmp iprange layer7
    PF_EXT_SLIB+=length limit mac mark multiport recent standard state quota
    -PF_EXT_SLIB+=tcp tcpmss time tos u32 udp web dscp connbytes webmon
    +PF_EXT_SLIB+=tcp tcpmss time tos u32 udp web dscp connbytes webmon string
    There may be a way to get xt-string into known modules at compile time, to avoid the need to "insmod", but it seems other such targets are activated by "modprobe" in the router rc files.

    I have great respect for Gargoyle/OpenWrt so have no doubt "weburl" works fine for http requests, but will have same limitations as tomato "web" for https.
     
    koitsu likes this.
  51. azdps

    azdps LI Guru Member

    thanks mstombs the string match now works. koitsu i compiled the lastest toastman's RT-N Ext build and made no changes except adding the xt_string match support. I'm was able to obtain the tcpdumps that you requested from above on both lan and wan. I will send you a pm so you can download the dump.

    iptables
    Code:
    root@RT-20CF30B74E90:/tmp# iptables -vL
    Chain INPUT (policy DROP 49 packets, 7910 bytes)
    pkts bytes target    prot opt in    out    source              destination
      91  4044 DROP      all  --  any    any    anywhere            anywhere            state INVALID
    2467 1216K ACCEPT    all  --  any    any    anywhere            anywhere            state RELATED,ESTABLISHED
        4  192 shlimit    tcp  --  any    any    anywhere            anywhere            tcp dpt:ssh state NEW
      32  2461 ACCEPT    all  --  lo    any    anywhere            anywhere
      43  3221 ACCEPT    all  --  br0    any    anywhere            anywhere
        0    0 ACCEPT    icmp --  any    any    anywhere            anywhere            limit: avg 1/sec burst 5
        0    0 ACCEPT    udp  --  any    any    anywhere            anywhere            udp dpts:33434:33534 limit: avg 5/sec burst 5
      30  9638 ACCEPT    udp  --  any    any    anywhere            anywhere            udp spt:bootps dpt:bootpc
     
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
    2062  811K            all  --  any    any    anywhere            anywhere            account: network/netmask: 192.168.1.0/255.255.255.0 name: lan
        0    0 ACCEPT    all  --  br0    br0    anywhere            anywhere
      90  3728 DROP      all  --  any    any    anywhere            anywhere            state INVALID
      283 13648 TCPMSS    tcp  --  any    any    anywhere            anywhere            tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU
    1023  177K restrict  all  --  any    vlan2  anywhere            anywhere
    1758  796K ACCEPT    all  --  any    any    anywhere            anywhere            state RELATED,ESTABLISHED
        0    0 wanin      all  --  vlan2  any    anywhere            anywhere
      127  6544 wanout    all  --  any    vlan2  anywhere            anywhere
      127  6544 ACCEPT    all  --  br0    any    anywhere            anywhere
     
    Chain OUTPUT (policy ACCEPT 2132 packets, 331K bytes)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain restrict (1 references)
    pkts bytes target    prot opt in    out    source              destination
    1023  177K rres00    all  --  any    any    anywhere            anywhere
     
    Chain rres00 (1 references)
    pkts bytes target    prot opt in    out    source              destination
      27  1296 DROP      all  --  any    any    anywhere            r-199-59-148-82.twttr.com
      30  1440 DROP      all  --  any    any    anywhere            r-199-59-148-10.twttr.com
      30  1440 DROP      all  --  any    any    anywhere            r-199-59-150-7.twttr.com
        0    0 DROP      all  --  any    any    r-199-59-150-7.twttr.com  anywhere
        0    0 DROP      all  --  any    any    r-199-59-148-82.twttr.com  anywhere
        0    0 DROP      all  --  any    any    r-199-59-148-10.twttr.com  anywhere
     
    Chain shlimit (1 references)
    pkts bytes target    prot opt in    out    source              destination
        4  192            all  --  any    any    anywhere            anywhere            recent: SET name: shlimit side: source
        0    0 DROP      all  --  any    any    anywhere            anywhere            recent: UPDATE seconds: 60 hit_count: 4 name: shlimit side: source
     
    Chain wanin (1 references)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain wanout (1 references)
    pkts bytes target    prot opt in    out    source              destination
    
    lsmod
    Code:
    root@RT-20CF30B74E90:/tmp# lsmod
    Module                  Size  Used by    Tainted: P
    xt_string                864  0
    ip6table_mangle          992  0
    ip6table_filter          704  0
    xt_recent              6800  2
    xt_IMQ                  736  0
    imq                    2320  0
    usblp                  11312  0
    ehci_hcd              34640  0
    leds_usb                2128  0
    led_class              1552  1 leds_usb
    ledtrig_usbdev          2464  1 leds_usb
    usbcore              114736  4 usblp,ehci_hcd,ledtrig_usbdev
    nf_nat_pptp            1440  0
    nf_conntrack_pptp      3808  1 nf_nat_pptp
    nf_nat_proto_gre        1072  1 nf_nat_pptp
    nf_conntrack_proto_gre    2464  1 nf_conntrack_pptp
    nf_nat_ftp              1568  0
    nf_conntrack_ftp        5792  1 nf_nat_ftp
    nf_nat_sip              5920  0
    nf_conntrack_sip      19008  1 nf_nat_sip
    nf_nat_h323            5504  0
    nf_conntrack_h323      37152  1 nf_nat_h323
    wl                  2610544  0
    dnsmq                  2032  1 wl
    et                    37344  0
    igs                    13584  1 wl
    emf                    17568  2 wl,igs
    
    access restriction
    [​IMG]
     
  52. koitsu

    koitsu Network Guru Member

    azdps, the iptable rules you have aren't correct.

    Code:
    Chain rres00 (1 references)
    pkts bytes target    prot opt in    out    source              destination
      27  1296 DROP      all  --  any    any    anywhere            r-199-59-148-82.twttr.com
      30  1440 DROP      all  --  any    any    anywhere            r-199-59-148-10.twttr.com
      30  1440 DROP      all  --  any    any    anywhere            r-199-59-150-7.twttr.com
        0    0 DROP      all  --  any    any    r-199-59-150-7.twttr.com  anywhere
        0    0 DROP      all  --  any    any    r-199-59-148-82.twttr.com  anywhere
        0    0 DROP      all  --  any    any    r-199-59-148-10.twttr.com  anywhere
    This is not using xt_string. Your last screenshot also confirms the same thing. These are iptable rules that are blocking IP addresses from a single DNS lookup at that moment in time; these do not match content of a packet. I already covered what this does and why it's wrong/broken/unreliable in this post:

    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-197153

    Please remove those two Access Rules you're using and instead do what was described here:

    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-196336

    Once you do that, **THEN** do the packet captures.
     
  53. azdps

    azdps LI Guru Member

    Changes made. koitsu see your pm for new tcpdumps.

    iptables
    Code:
    root@RT-20CF30B74E90:/tmp/home/root# iptables -L -v -n
    Chain INPUT (policy DROP 31 packets, 1837 bytes)
    pkts bytes target    prot opt in    out    source              destination
      93  4378 DROP      all  --  *      *      0.0.0.0/0            0.0.0.0/0          state INVALID
    2172 1223K ACCEPT    all  --  *      *      0.0.0.0/0            0.0.0.0/0          state RELATED,ESTABLISHED
        4  192 shlimit    tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:22 state NEW
        3  567 ACCEPT    all  --  lo    *      0.0.0.0/0            0.0.0.0/0
      113  8747 ACCEPT    all  --  br0    *      0.0.0.0/0            0.0.0.0/0
        0    0 ACCEPT    icmp --  *      *      0.0.0.0/0            0.0.0.0/0          limit: avg 1/sec burst 5
        0    0 ACCEPT    udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpts:33434:33534 limit: avg 5/sec burst 5
      54 17477 ACCEPT    udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp spt:67 dpt:68
     
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
      135  111K REJECT    all  --  *      *      0.0.0.0/0            0.0.0.0/0          STRING match "twitter.com" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    141K  172M            all  --  *      *      0.0.0.0/0            0.0.0.0/0          account: network/netmask: 192.168.1.0/255.255.255.0 name: lan
        0    0 ACCEPT    all  --  br0    br0    0.0.0.0/0            0.0.0.0/0
      41  1903 DROP      all  --  *      *      0.0.0.0/0            0.0.0.0/0          state INVALID
      900 44404 TCPMSS    tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp flags:0x06/0x02 TCPMSS clamp to PMTU
    140K  172M ACCEPT    all  --  *      *      0.0.0.0/0            0.0.0.0/0          state RELATED,ESTABLISHED
        0    0 wanin      all  --  vlan2  *      0.0.0.0/0            0.0.0.0/0
      485 25045 wanout    all  --  *      vlan2  0.0.0.0/0            0.0.0.0/0
      485 25045 ACCEPT    all  --  br0    *      0.0.0.0/0            0.0.0.0/0
     
    Chain OUTPUT (policy ACCEPT 1546 packets, 230K bytes)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain shlimit (1 references)
    pkts bytes target    prot opt in    out    source              destination
        4  192            all  --  *      *      0.0.0.0/0            0.0.0.0/0          recent: SET name: shlimit side: source
        0    0 DROP      all  --  *      *      0.0.0.0/0            0.0.0.0/0          recent: UPDATE seconds: 60 hit_count: 4 name: shlimit side: source
     
    Chain wanin (1 references)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain wanout (1 references)
    pkts bytes target    prot opt in    out    source              destination
    
     
  54. koitsu

    koitsu Network Guru Member

    Thank you -- that looks correctly configured. I will analyse the captures now.
     
  55. koitsu

    koitsu Network Guru Member

    I've reviewed packet captures azdps provided me, with the xt_string module in use to block packets containing the string "twitter.com". It took me a little while to analyse the captures because there's other SSL traffic in there for sites like Facebook and others happening at the same time, so I had to narrow things down to the 199.59.148.0/22 network block.

    The results are inconclusive at this time.

    The WAN packet capture was done too late. I can tell from the timestamps in the WAN capture -- the packets are almost 2 minutes after that of the LAN capture. This is more or less useless.

    I have wasted enough time on this, I think. I would rather just do the capturing myself (no offence intended to people, but I know what I'm doing and it's just easier if I do it myself). However, xt_string is not a built/provided module on Toastman firmware tomato-K26USB-1.28.0500.5MIPSR2Toastman-RT-N-Ext.trx so I do not have a way to test this.

    If someone is able to use git to pull down that Toastman release (pay very close attention to which it is!) and build the xt_string.ko module + provide it to me (as an attachment here, etc.) so I can load it with insmod and then test myself, I would be grateful. Remember: it needs to be built for that specific firmware version. Thank you.
     
  56. azdps

    azdps LI Guru Member

    koistu the wan capture was done completely separate from the lan capture. i started with the lan capture and did the lan tcpdump like you described it should be done. i then did the wan capture about 2 minutes later attempting to access the twitter website once again starting from scratch just for the wan capture. anyways i will build the firmware you need.
     
  57. koitsu

    koitsu Network Guru Member

    I needed both the WAN and LAN capture done at the same time. I explained this quite concisely in my post ("two captures need to be done simultaneously"):

    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-196009

    Also I appreciate the offer of a firmware, but I'm not going to run/change my firmware to test this. It's just not feasible and it impacts other things I have going on. All I need is the xt_string.ko module built for tomato-K26USB-1.28.0500.5MIPSR2Toastman-RT-N-Ext.trx to be able to test. :)
     
  58. koitsu

    koitsu Network Guru Member

    Been discussing this privately with azdps.

    Turns out I'm wrong, and I believe mstombs touched base on this earlier in the thread but I had completely forgotten.

    Getting the module by itself won't work like I expected -- there are netfilter-related code pieces/functions that need to be available within netfilter itself for this module to work. Otherwise when loading xt_string.ko it complains of missing/undefined symbols.

    Code:
    root@gw:/tmp# insmod xt_string.ko
    insmod: can't insert 'xt_string.ko': unknown symbol in module, or unknown parameter
    
    More specifically, it's complaining that xt_string uses netfilter functions called skb_find_text(), textsearch_destroy(), and textsearch_prepare(), which don't exist in the netfilter code, hence the error. Proof:

    Code:
    root@gw:/tmp# nm xt_string.ko
    00000000 D __this_module
    00000000 T cleanup_module
    00000000 T init_module
    U skb_find_text
    U textsearch_destroy
    U textsearch_prepare
    U xt_register_matches
    U xt_unregister_matches
    
    All modules use xt_register_matches() and xt_unregister_matches() (I verified this against xt_web.ko, just as a comparison), so the above functions aren't provided by the netfilter module in the kernel.

    So basically for me to test this, I do need a custom build of tomato-K26USB-1.28.0500.5MIPSR2Toastman-RT-N-Ext.trx that has this netfilter capability exported -- and while I'm there I can build xt_string myself.

    I will have to spend a lot of time to figure out how to build Toastman firmwares on my own. I'll get to this when I have time / when I have the interest.
     
  59. azdps

    azdps LI Guru Member

    koitsu here are easy instructions for you to build tomato firmware.
     

    Attached Files:

    Dinnn likes this.
  60. koitsu

    koitsu Network Guru Member

    Thank you azdps, this is what I was looking for. I managed the git clone/etc. on my own, but the actual building process was seriously busted and repairing that was a task in itself. :) This will help me get started. Again thank you very much.
     
  61. koitsu

    koitsu Network Guru Member

    I wish somewhere, in all of this "documentation", all these READMEs, all this instructional stuff on forums, etc. etc... someone had bothered to mention that you need to use an i386 (32-bit) Linux distribution for the toolchain binaries stored in git (specifically tools/brcm/hndtools-mipsel-linux/bin/mipsel-linux-uclibc-gcc-4.2.4) to work.

    Using amd64 (64-bit) results in "No such file or directory", which isn't very intuitive; I cannot imagine how a generic end-user would have determined this on their own.

    I'm sure they would work just fine if I could rebuild the toolchain, but I can't find any official documentation on that either. *shakes head*

    Anyway, finally getting somewhere...
     
  62. koitsu

    koitsu Network Guru Member

    And further information:

    The gperf package also needs to be installed on the system, otherwise you'll experience failures when building the minidlna package (specifically library libid3tag) as documented here.

    So the "sudo apt-get install" instructions in that .txt file should be updated to reflect the need for the "gperf" package.

    And while someone's there, they should also remove the unnecessary package installation duplicate of libncurses5-dev (it's in there twice, look carefully).
     
  63. azdps

    azdps LI Guru Member

    the instructions on how to compile tomato firmware has been updated to reflect koitsu suggested changes above.
     

    Attached Files:

    Dinnn likes this.
  64. mstombs

    mstombs Network Guru Member

    @koitsu - you can build tomatousb on 64-bit linux with git toolchain binaries, I do, there must be some x86 lib32 libs needed to get apt-get 'ted (but too long ago for me to recall). The toolchain can be re-built natively, see the "build26.sh" in the toolchain folder - but warning it's a version of an OpenWrt builder that 'downloads half the internet' to get the original kernel and binutils sources so It can take all night then fail on something small...
     
  65. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

  66. koitsu

    koitsu Network Guru Member

    Thanks. I should note those build instructions are also missing gperf; might want to update that too.

    I'm not under the impression that this is a library issue. A missing or incompatible library should return an error from ld.so or dl*() related functions, not "No such file or directory". On the other hand, an ELF binary built for use on i386, when attempted to be run on amd64 (whose kernel lacks 32-bit binary emulation/support), would return the error I described. This is why I say I don't think it's a 32-bit library shim thing.

    Regardless, I have no problem using an i386 distribution in the least, especially since this is just a VM dedicated to building a firmware. :) Thanks.
     
  67. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Don't know they are missing anything. I follow those build instructions to the letter and they work for me.
     
  68. lancethepants

    lancethepants Network Guru Member

  69. koitsu

    koitsu Network Guru Member

  70. mstombs

    mstombs Network Guru Member

    I think another packages "ia32-libs" maybe needed on 64 bit, but trouble with this sort of info is that it all depends on what is installed by default or by other things. I also gave up on Ubuntu and moved to Linux Mint Debian at some point because things worked more easily (and desktop was more usable!). There used to be a big dependence on the Linux shell in use (bash vs dash) and versions of make tools, but there have been steady improvements in git - thanks to all the active mod developers.
     
  71. koitsu

    koitsu Network Guru Member

    Folks following the thread, please read everything I've written here slowly and in full. Re-read if need be; it's a lot, I know.

    I've managed to successfully build things on Debian 6.0.6 (i386). I should note that on Debian, sudo does not come with the system, so adding sudo to the "apt-get install" instructions would probably be wise (it will be a no-op on things like Ubuntu where sudo comes pre-installed).

    I HAVE gotten a firmware built, but I haven't had time to test it. Gotten very little sleep due to being in bad health, and I have a dental appointment today. Big thanks to azdps for giving me those instructions. I plan on making a blog post documenting how to get the firmware to build at a future time.

    And now for something completely different...

    The issue of xt_string.ko not getting loaded dynamically (by modprobe, etc.) is indeed odd; I don't have an explanation for that. That's my own ignorance, not the fault of the software. The easiest solution by far is to compile xt_string support into netfilter natively rather than as a module. That's as simple as:

    Code:
    CONFIG_NETFILTER_XT_MATCH_STRING=y
    (Note the value "y" not "m")

    Overall this adds a measly 5KBytes or so to the firmware size. I think that's acceptable.

    I should also note that xt_string module IPv6 support doesn't appear to exist/work. I'm not completely sure how to fix that; adding it to PF6_EXT_SLIB is not the correct solution; that results in a library called ip6ipt_string trying to be built (if I remember right) and there's no such source. Is it fixable? Probably, but I haven't spent the time to figure it out yet.

    Finally:

    About the iptable rule used for using -m string -- we should really talk more about this at length. I got to thinking about it last night in depth and there are a bunch of caveats. For example, it should be modified to only match against TCP port 443 (so use arguments -p tcp --dport 443) since otherwise if any other kind of plaintext TCP packet happened to contain the string "twitter.com" within the first 600 bytes of the packet payload, it would be blocked as well. Users would find that they couldn't mention twitter.com in an Email they send out, for example. That sort of thing -- and we don't want to block that kind of traffic.

    For blocking HTTP-based methods, I think the string to block would be "Host: twitter.com" (keying off the HTTP Host: header submit by the browser), and maybe also add -p tcp --dport 80. There's also the issue of the byte search length being between bytes 1-600 of the payload; sometimes there are lots and lots of HTTP headers included by a client (more than 600 bytes worth) and the Host: header could come *after* those 600 bytes, thus wouldn't be blocked.

    Finally, for both above cases, we should really be matching against very specific TCP flags if at all possible, or at a certain point/phase of TCP sequences. I don't know if this is possible, but I think it is.

    For HTTPS, we'd want to specifically key off of the first PSH+ACK I think -- I need to look at packet captures to make sure. Basically the idea is that we only want to match the block against a very specific "flow" of packets, e.g. SYN, SYN+ACK, ACK, ACK, PSH+ACK (SSL Client Hello) <-- block here.

    For HTTP, we'd want to specifically key off of the first payload ACK, I think -- again, need to look at packet captures to make sure. I.e. SYN, SYN+ACK, ACK, ACK (GET/POST/HEAD request with Host: header) <-- block here.

    You want to try and be a specific as possible when doing this kind of string-based layer 7 filtering, because there are so many edge cases that could get impacted from a badly-written rule.
     
  72. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    Anyone have anything new on this?
     
  73. samjones3

    samjones3 Serious Server Member

    Koitsu et al:

    This thread discusses something I have great need for: SSL blocking (https://facebook.com, https://youtube.com, etc).

    It sounds very promising!

    What is the status and next step?

    Thank you!
     
  74. koitsu

    koitsu Network Guru Member

    Have no news to report. Have not had time/interest to get xt_match_string working. I have built a firmware with it included (I chose to include it statically (meaning value of "y" not "m")) but I have not had time/interest to re-flash my only working router with it. If I brick it I'll be without Internet access for a while, and I can't risk that.

    Folks wanting to give it a try can build the firmware themselves and make the adjustments shown here in these two posts:

    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-198099
    http://www.linksysinfo.org/index.ph...ction-block-https-websites.45988/#post-197539

    Attached is the diff I made when trying it out (that is to say, building+compiling with success -- whether or not it functionally works is unknown, but I trust mstombs!)

    Basically bottom line (as I see it): the folks who want this need to put in the time/cycles to get it working. I know, that's not the response a lot of people want, but it's true in this case. I'm just a simple caveman engineering sorta guy... :)
     

    Attached Files:

  75. Europan9600

    Europan9600 Networkin' Nut Member

    Hi WRD,

    Looks like you used some method to block accessing https addressed sites, in EasyTomato 0.8 . Which method did you use?

    This whole thread was a great discussion btw. Amazing input by everyone! So much learned in this thread.

    Regards,
     
  76. WRD - EasyTomato

    WRD - EasyTomato Networkin' Nut Member

    I don't remember where the commits are off the top of my head, but we did a string search to go after the DNS lookups. I think we also look at SSL handshakes, but the text in the cert often doesn't match the domain you're trying to block, so that doesn't really add much. (for example gmail.com's cert is accounts.google.com)
     
  77. DJF - EasyTomato

    DJF - EasyTomato Serious Server Member

  78. Europan9600

    Europan9600 Networkin' Nut Member

    Great. Thanks, I will look it up. I just realized that blocking twitter doesn't seem to block https access to twitter (while facebook is blocked via http and https). I guess twitter cert text and the domain text differ.
     
  79. koitsu

    koitsu Network Guru Member

    Please do not spread misinformation/speculative guesswork. Instead, take the time to look at it; capture some packets on port 443 during initial TCP setup and so on and look at the Client Hello TLSv1 request. You'll find that the SNI is platform.twitter.com (don't ask me where that comes from); this is in the outbound request going to twitter, not the response coming back from them, and is around the 240 byte mark. The --string twitter.com would match that as well, since the xt_string module matches anywhere within the first 600 bytes of payload (it's not a regex), however I believe the iptables rules may be only looking at response packets (not sure, this is speculative on my part :) ).

    My guess is that the small payload byte range (bytes 1 to 600). The payload for TLSv1 Server Hello (not client) is extremely long for twitter -- around 3-4KBytes -- because of the long certificate chain twitter chooses to use; the relevant Server Hello string comes back at around the 880 byte mark.

    The problem with extending the size (from 1 to 600 bytes, to something like 1 to 1400 bytes) is that it increases CPU load and greatly increases the risk of false positives. This is one of the many reasons why using the xt_string module for this kind of blocking can be very dangerous.

    I really wish people wanting SSL blocking would take the time to actually understand the complications/how this works and why it's not always reliable. (Not talking about the EasyTomato guys -- you guys are fine. :) )

    I cannot help past this point.
     
  80. Europan9600

    Europan9600 Networkin' Nut Member

  81. noyp

    noyp Network Guru Member

    disregard, i already found the old post regarding exemptions


    hi koitsu,
    can you give a sample rule to exempt a single ip from https blocking, mine below didnt work

    Code:
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    num   pkts bytes target     prot opt in     out     source               destination       
    1    24261 1398K ACCEPT     all  --  br0    *       10.1.1.123           0.0.0.0/0         
    2     2503 1373K REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "facebook.com" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    3      261  230K REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           STRING match "t


    thanks
    noyp
     
    Last edited: Feb 19, 2014
  82. JayNort

    JayNort Network Newbie Member

    This worked for me

    iptables -t nat -I PREROUTING -m tcp -p tcp -d www.twitter.com --dport 443 -j DROP
     
  83. koitsu

    koitsu Network Guru Member

    @JayNort That will not work correctly 100% of the time. DNS resolution of www.twitter.com is done at the time you run iptables. And here's how twitter's DNS works:

    www.twitter.com is a CNAME to twitter.com. The TTL on the CNAME is 600 seconds.

    twitter.com returns three A records (i.e. they're round-robin): 199.59.150.39, 199.59.149.230, and 199.59.149.198. All those A records have a TTL of 30 seconds.

    TL;DR -- what you propose will not work reliably.

    Next, there's the fact that blocking twitter.com or www.twitter.com doesn't guarantee that you'll block content hosted at Twitter, or at a Twitter CDN. Those are probably all under different network blocks, or have their own domain or FQDN associated with them (CDNs are notoriously annoying in this regard).
     
    Last edited: Jun 10, 2014
    HitheLightz likes this.
  84. yasavvy

    yasavvy Reformed Router Member

    Hi guys so can we get some answers without having to earn a PhD in networking? We just want to block https facebook, twitter, or youtube, or all 3. Preferably with IPtables in Shibby Tomato firmware. Hi. Not degree in networking please. Thanks.
     
  85. shibby20

    shibby20 Network Guru Member

    try add do firewall script:

    iptables -I FORWARD 1 -m string --string twitter --algo bm --from 1 --to 600 -j REJECT
    iptables -I FORWARD 1 -m string --string youtube --algo bm --from 1 --to 600 -j REJECT
    ...
     
    HitheLightz likes this.
  86. Monk E. Boy

    Monk E. Boy Network Guru Member

    Another option is to create DNS records for twitter, facebook, twitter, etc. in Tomato's DNS server, then block the client from connecting to another DNS server.
     
  87. koitsu

    koitsu Network Guru Member

    This would only work if dnsmasq (re: Tomato's DNS server) supports wildcard record matching, i.e. facebook.com and *.facebook.com -- does it? I'm not aware of it supporting this. (I implement this methodology by running an actual DNS server (BIND) on my FreeBSD box on my LAN, to deal with ad blocking for Skype).

    It also doesn't solve the issue of places that use CDNs, like Facebook (fbcdn.com / fbcdn.net -- most people don't know about that), nor if someone later introduces use of a CDN into their infrastructure. It's one of those cat-and-mouse games that requires anyone wanting to do content blocking to constantly be checking things out (because surely those who are to be blocked aren't going to tell the admin "hey uh you know i can get to facebook if i use a magic URL...")
     
  88. mstombs

    mstombs Network Guru Member

    Yes dnsmasq does support domain blocking "--address=/<domain>" was used in early adblocking schemes through dns poisoning, but I notice latest scripts seem to have gone back to more selective hosts file method.

    But this doesn't block direct use of ip addresses, or dnscrypt to bypass the router dns server. shibby's rules above may interfere with https certificate interchange, if placed in INPUT or OUTPUT chain could interfere with dns lookups (as easytomato)
     
  89. Monk E. Boy

    Monk E. Boy Network Guru Member

    Yup, dnsmasq can easily handle full wildcards, and obviously you would not only need to block the main domains but related (CDN, etc.) domains as well. To be honest most of the requisite information is readily available since it's published for enterprise environments, to prevent them from blocking access. As for the "hey if you use this magic URL you can get in" cases, you'd be amazed something as simple as logging DNS queries uncovers those magic URLs once as they spread beyond a couple individuals.

    As with all forms of restrictions, given a determined enough individual there's no stopping them. It's all about putting enough stumbling blocks in their path that most people will give up or get caught. Airport security, anyone?

    Sometimes, though, you'll need to prove that you've taken the necessary steps to legally defend yourself in court. If you've taken steps to prevent K12 students from downloading porn while in school, and they find a way to start downloading porn, you could end up walking a short plank unless you can show that you've made a best faith effort to restrict them. And while short plank is somewhat of an dramatization, the penalties are rather severe... termination, fines, even jail/prison time if you're particularly unlucky (also known as prosecutorial misconduct).

    In those cases, though, these restrictions are totally inadquate. Simply block all direct access to the internet from client devices and make them access content through proxy servers, which is where restrictions get implemented. Hosts files & IP addresses are worthless at that point (assuming a proxy server requires a FQDN to get passed to it). The problem then becomes living with the limitations of not being able to directly connect to the internet. No skype, no spotify, no netflix, no fun, just the assignment they were assigned to work on in class, oh bother.
     
  90. minos

    minos Reformed Router Member

    Tried to block Facebook (and others websites for testing) with Access Restriction tool, but only http is blocked, not https

    sadly :'(

    I'm sure on previous Shibby build (<121 K24)it was ok to block https access like http
    or I'm not remembering well...?
     
  91. kthaddock

    kthaddock Network Guru Member

    [RELEASE] 119
    All version:
    - Access Restriction: return to old „web module” style. „String” module is still in image, advanced users can use it in own iptables rules,
     
  92. minos

    minos Reformed Router Member

    Humm, kk thx for replying.
    Do you know if there is an explanation why using an "old" feature... less efficient (I think)?

    edit: I've found this comment where you looks like happy to have this revert... humm, what was the problem the this String Module that can be able to block https? lol I'm a little noob about kernel and devs things :p Tomato Shibby's Releases
     
    Last edited: Jul 18, 2014
  93. kthaddock

    kthaddock Network Guru Member

    There is no problem with string module. It has some quirks but have to use with cautious.
    Why destroy Access Restriction with alot of function for one module.
    It's easy to start string module with insmod and put iptables rules into firewall box or in a file.
     
    Last edited: Jul 18, 2014
  94. jerrm

    jerrm Network Guru Member

    Web allows for some basic regular expressions matching, string doesn't. Web matches specifically on http urls, string matches on the first 600 bytes of the packet. String can overmatch or undermatch.

    Web can not match on https because the packet is encrypted. String works on https because it looks at the first 600 bytes of packets before encryption is set up and IF portions of certificate negotiation match, the packet is blocked. The problem is that doesn't always work on more complicated chains. The rules as generated by the Tomato also blocked dns queries if the string matches.

    I never looked too closely at the counters, but my guess is the vast majority of the string based blocking actually occurred because the dns query was blocked, and the better way to do that is with dnsmasq.
     
  95. koitsu

    koitsu Network Guru Member

    One clarification point: at least on Shibby (not sure about others), the firewall rules used alongside xt_string only match against TCP ports 80 and 443, so DNS traffic (UDP port 53, unless using AXFRs in which case TCP port 53 is used as a fallback) wouldn't be impacted.

    I cannot remember if the "original/old" Access Rules model (using web) used iptables flags like -p tcp -m tcp --dport 80 or not (I'm betting it didn't, though the underlying web module code itself may actually look at TCP headers to check certain ports -- or maybe it doesn't intentionally so that it could be used to block servers on alternate ports, ex. http://somesite:4000/ )

    While xt_string "works" for some people, I still stand firm that it's a bad idea to have put this into place without a full revamp of the Access Restrictions section, and using different NVRAM variables for matches. The reason is that the old web module allowed a basic form of regex, while string does not, and it has already come up twice here on this forum where Access Restrictions stopped working because the user used regex in their rules + upgraded their firmware to one which used xt_string.

    Honestly, I think the best approach would be to use both modules together (web and xt_string), specifically using web to match against standard HTTP stuff, and use xt_string with -p tcp -m tcp --dport 443 specifically (to minimise damage / only look at SSL). For SSL servers running on a port other than TCP port 443, you're screwed aside from writing your own iptables rules.
     
  96. jerrm

    jerrm Network Guru Member

    No. Shibby also looked at 53 both udp and tcp. From Shibby 117:
    Code:
    Chain restrict (2 references)
    pkts bytes target prot opt in out source destination
    1739 151K rres01 all -- * * 0.0.0.0/0 0.0.0.0/0
    Chain rres01 (1 references)
    pkts bytes target prot opt in out source destination
    1666 144K rstr01 tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 53,80,443
    34   2141 rstr01 udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:53
    Chain rstr01 (2 references)
    pkts bytes target prot opt in out source destination
    0 0 REJECT udp -- * * 0.0.0.0/0 0.0.0.0/0 STRING match "bogus.domain.com" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 STRING match "bogus.domain.com" ALGO name bm FROM 1 TO 600 reject-with tcp-reset 
    The rules do not restrict the port.
    Code:
    Chain restrict (2 references)
    pkts bytes target prot opt in out source destination
    671 101K rres01 all -- * * 0.0.0.0/0 0.0.0.0/0
    Chain rres01 (1 references)
    pkts bytes target prot opt in out source destination
    11 6801 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 web --hore "portquiz.net" reject-with tcp-reset
    Chain shlimit (2 references)
    pkts bytes target prot opt in out source destination
    0 0 all -- * * 0.0.0.0/0 0.0.0.0/0 recent: SET name: shlimit side: source
    0 0 logdrop all -- * * 0.0.0.0/0 0.0.0.0/0 recent: UPDATE seconds: 60 hit_count: 4 name: shlimit side: source
    
    The module doesn't appear to either. http://portquiz.net/ and http://portquiz.net:4000/ will both be blocked.

    Basically what I suggested too.
     
  97. koitsu

    koitsu Network Guru Member

    Eesh, I didn't know it looked at UDP/TCP ports 53! Well, I guess technically that's one thing xt_string can do, of course, but I too worry about high numbers of false positives (especially if someone was to just enter "foo.com" or something, since that'd also match against blahfoo.com).

    I think getting what's needed out of xt_string really requires a large Access Restrictions GUI revamp, which I'm pretty sure earlier in this thread (first few pages) I pointed out / advocated the need for if xt_string was used. Heh... I've come full circle! ;D
     
  98. minos

    minos Reformed Router Member

    Interesting answers, many thx ;)
     
  99. Magdiel1975

    Magdiel1975 Networkin' Nut Member

    That is my thought exaclty, lol - I can tell these guys are very knowledgable on the matter, but us simpleton creatures do not understand any of it :)

    So, guys.. just to clarify, there is not iptable or some sort of code that would block both http and htttps?
    thanks for your time.
     
  100. mstombs

    mstombs Network Guru Member

Share This Page