1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Port mirroring

Discussion in 'Tomato Firmware' started by humba, Oct 20, 2007.

  1. humba

    humba Network Guru Member

    Does anybody know if it's possible to activate port mirroring on the LAN switch?
     
  2. mikester

    mikester Network Guru Member

    yes its possible
     
  3. humba

    humba Network Guru Member

    Any idea how to do it?
     
  4. mikester

    mikester Network Guru Member

    It all boils down to what are you trying to do....

    Are you trying to read peoples email or simply monitor network traffic?

    Reconstructing packet streams would involve marking specified packets and routing them through a sniffer running ethereal/wireshark.

    Try googling "port mirror iptables".

    If you just want to log traffic patterns simply turn on syslog (~/admin-log.asp) and run a utility on your PC like wallwatcher.
     
  5. humba

    humba Network Guru Member

    I don't care about emails, but SIP signaling and in some instances RTP. From the manpage, I don't think the mirror option in iptables is what I'm looking for.. I don't want to touch the packets.. I want to take everything coming in and going out on port X (could be LAN, could be WAN, could be wireless) to another port.. like a trusty old hub would do it, or a switch with port mirroring function.
     
  6. ref38

    ref38 Addicted to LI Member

    Mikester - since you appear to know...

    I also want to do port mirroring on my WRT54GL running Tomato 1.21.

    I am not looking to monitor anyone's e-mail but do want to run an intrusion detection system (psad) and bothunter. To be able to do that, I need a monitor port that sees all the traffic between my DSL modem and the WRT.

    I have a spare port and a spare computer that can run the software, and I could just slap a hub in between the modem and the WRT, but that would throttle it down to half duplex and I don't want to do that.

    I googled mirroring with iptables and I found conflicting information about --tee command. Is that the one you are referring to?

    Thanks in advance,
    Robert
     
  7. humba

    humba Network Guru Member

    I guess --tee would be the option but I have no idea on how to use it.
    I went with a different approach: got myself an 8 port managed layer2 switch (more expensive than the router but well worth it) and set up port mirroring there and place it before the router..
     
  8. mikester

    mikester Network Guru Member

    I was trying to go down the same path you are looking at.

    One method is to tag all the packets and re-route them.

    I found it easier and quicker to install an IPCOP box with NTOP and wireshark than to fiddle with the Tomato image.
     
  9. ref38

    ref38 Addicted to LI Member

    Hey guys, thanks for the information!

    I don't know but would think designating a port as a monitoring port and mirroring packets between the WAN port and the monitoring port would be something better done in the firmware itself.

    I've been experimenting with the logging options and found it was pretty easy to log to a CIFS mount from my NAS but I wasn't able to figure out a way to get it to persist through reboots. The scripts don't seem to fire at the right times. Still learning though. There probably is a good way.

    The alternative is to just UDP the logs out to a box that has its syslog listening. That should work just as good and the box could log to the NAS just as easily.

    But none of this gets the packets themselves for deeper inspection. Just the iptables log messages.

    I could just rearrange a bit and put a 2-port linux box between the modem and router but I like the idea of the linux box being on a monitoring port instead of actively involved in the transactions. Plus its performance won't impact the data flow if it is just monitoring. I also have the modem and router at the service entrance and putting a linux box there wouldn't be so easy. Same for a managed switch but less so.

    I'll have to think about this and see if I can find out more about the --tee option.

    Thanks again, though!
     
  10. ref38

    ref38 Addicted to LI Member

    One more thing - I know that just looking at iptables log messages is pretty lightweight duty. But if doing reassembly of packets and deep inspection it gets more involved. The box I want to use is also a bit anemic anyway. ;-)
     
  11. wildbill

    wildbill Addicted to LI Member

    I'd also like to forward all packets on one interface to a particular (h/w) port on the router...have been researching for a couple of days and concluded it would take me many weeks to begin to understand iptables and the "ip" commands that seem to be necessary. So I can understand why humba just used a hub! [correction: a managed layer2 switch.] But I want the mirrored port on the inside of the firewall, so a hub on the outside is not exactly the solution I'm looking for.

    Have located a post here that gives an example of how to do it, but it uses a "-j ROUTE" switch that is not implemented in my version of tomato.

    Further research seems to reveal that routing is not really iptables' job, so it needs to be done with an "ip" rule. I think.
     
  12. mstombs

    mstombs Network Guru Member

    I think you need to get a mod author interested, the "ROUTE tee" target needs to be compiled as an installable kernel module so you can use it. iptables, ifconfig, route and ip are all userspace apps that configure the kernel, there is often more than one way to achieve the same result!
     
  13. rhester72

    rhester72 Network Guru Member

    Working on route tee :)

    Rodney
     
  14. rhester72

    rhester72 Network Guru Member

    Coming soon to a Tomato router near you:

    Code:
    # iptables -j ROUTE -h
    iptables v1.3.7
    
    Usage: iptables -[AD] chain rule-specification [options]
           iptables -[RI] chain rulenum rule-specification [options]
           iptables -D chain rulenum [options]
           iptables -[LFZ] [chain] [options]
           iptables -[NX] chain
           iptables -E old-chain-name new-chain-name
           iptables -P chain target [options]
           iptables -h (print this help information)
    
    Commands:
    Either long or short options are allowed.
      --append  -A chain            Append to chain
      --delete  -D chain            Delete matching rule from chain
      --delete  -D chain rulenum
                                    Delete rule rulenum (1 = first) from chain
      --insert  -I chain [rulenum]
                                    Insert in chain as rulenum (default 1=first)
      --replace -R chain rulenum
                                    Replace rule rulenum (1 = first) in chain
      --list    -L [chain]          List the rules in a chain or all chains
      --flush   -F [chain]          Delete all rules in  chain or all chains
      --zero    -Z [chain]          Zero counters in chain or all chains
      --new     -N chain            Create a new user-defined chain
      --delete-chain
                -X [chain]          Delete a user-defined chain
      --policy  -P chain target
                                    Change policy on chain to target
      --rename-chain
                -E old-chain new-chain
                                    Change chain name, (moving any references)
    Options:
      --proto       -p [!] proto    protocol: by number or name, eg. `tcp'
      --source      -s [!] address[/mask]
                                    source specification
      --destination -d [!] address[/mask]
                                    destination specification
      --in-interface -i [!] input name[+]
                                    network interface name ([+] for wildcard)
      --jump        -j target
                                    target for rule (may load target extension)
      --goto      -g chain
                                  jump to chain with no return
      --match       -m match
                                    extended match (may load extension)
      --numeric     -n              numeric output of addresses and ports
      --out-interface -o [!] output name[+]
                                    network interface name ([+] for wildcard)
      --table       -t table        table to manipulate (default: `filter')
      --verbose     -v              verbose mode
      --line-numbers                print line numbers when listing
      --exact       -x              expand numbers (display exact values)
    [!] --fragment  -f              match second or further fragments only
      --modprobe=<command>          try to insert modules using this command
      --set-counters PKTS BYTES     set the counter during insert/append
    [!] --version   -V              print package version.
    
    ROUTE target v1.11 options:
        --oif       ifname          Route packet through `ifname' network interface
        --iif       ifname          Change packet's incoming interface to `ifname'
        --gw        ip              Route packet via this gateway `ip'
        --continue                  Route packet and continue traversing the
                                    rules. Not valid with --iif or --tee.
        --tee                       Duplicate packet, route the duplicate,
                                    continue traversing with original packet.
                                    Not valid with --iif or --continue.
    
    Rodney
     
  15. wildbill

    wildbill Addicted to LI Member

    Cool!

    PS Not knowing how these binaries work...I'm actually using a non-Linksys router...hope that doesn't disqualify me!

    --Bill
     
  16. rhester72

    rhester72 Network Guru Member

    Bill,

    Tell me exactly what version (regular or ND, if a mod which one, and what version number) of Tomato you're running and I will get a test binary to you. Works well on my WRT54GL, but I'd rather not release the patches into the wild until you have verified as well.

    Rodney
     
  17. wildbill

    wildbill Addicted to LI Member

    I'm running the standard distro "v1.23.1607 Built on Sun, 14 Dec 2008 02:54:59 -0800" on a Buffalo whr-hp-g54.
     
  18. rhester72

    rhester72 Network Guru Member

    Bill,

    Give this a try:

    http://multics.dynalias.com/tomato/Tomato_1_23_ROUTE+tee.7z

    You will need to set up the iptables rule(s) yourself manually - since I'm not clear on whether you are attempting a full port span, replication based on source or dest port, replication based on source or target IP, etc., I can't really suggest the exact iptables rule you should set up. The code has been lightly tested and seems to work (and should since it's based on precisely the same code that has been used for this for ages), but I'd rather you put it through its paces a bit more before I consider submitting it for inclusion in mainline Tomato.

    Rodney
     
  19. wildbill

    wildbill Addicted to LI Member

    Rodney

    Sent you a PM.

    Thanks
     
  20. wildbill

    wildbill Addicted to LI Member

    Okay, I'm going to show my ignorance now.

    Let's say I want to send all tcp traffic to the machine at ip address 10.1.1.1.

    I try these commands:

    iptables -t mangle -A PREROUTING -p tcp -j ROUTE --gw 10.1.1.1 --tee
    iptables -t mangle -A POSTROUTING -p tcp -j ROUTE --gw 10.1.1.1 --tee​

    But I get back:

    iptables: No chain/target/match by that name​

    Interestingly, this does not error:

    iptables -t mangle -A PREROUTING -p tcp​

    (not that it does anything)

    Edit:

    At least part of the answer is that I needed to do this:

    insmod ipt_ROUTE​
     
  21. wildbill

    wildbill Addicted to LI Member

    Okay, I've got the traffic from one IP to mirror to another, but the source machine does not get any replies.

    The commands I used:

    iptables -t mangle -A PREROUTING -p tcp --source 10.1.1.2 -j ROUTE --gw 10.1.1.1 --tee
    iptables -t mangle -A POSTROUTING -p tcp --source 10.1.1.2 -j ROUTE --gw 10.1.1.1 --tee​

    In other words, when I start a browser session from 10.1.1.2, it times out waiting for a reply. I was under the impression that the --tee option was supposed to allow10.1.1.2 to "work," that is, exchange regular traffic.
     
  22. rhester72

    rhester72 Network Guru Member

    Try it with only the PREROUTING chain and leave POSTROUTING alone. My tests with Wireshark showed that to be sufficient on my setup.

    Rodney
     
  23. wildbill

    wildbill Addicted to LI Member

    Nope, getting the same result.
     
  24. mstombs

    mstombs Network Guru Member

    You would need to ensure that the sniffing machine 10.1.1.1 does not reply to any requests from 10.1.1.2 don't you? Is the setting of the WAN IP local loopback important?
     
  25. wildbill

    wildbill Addicted to LI Member

    mstombs, I'm not smart enough to translate your comment into something resembling specific commands or configuration changes...would you be so kind as to give another clue to the clueless?

    Edit: In the router, I added a route to a to a non-existent IP address on the destination machine (using its real IP as the "gw" in the route command)...same result again...
     
  26. lovingHDTV

    lovingHDTV LI Guru Member

    wildbill,
    Any updates? I've been waiting for this capability for a long time to allow me to get Vonage SIP packets for callerID purposes mirrored to a machine that can make use of them.

    thanks,
    dave
     
  27. wildbill

    wildbill Addicted to LI Member

    Sorry mate, :frown: I was never able to get it working, despite the excellent help from rhester72. I was probably doing something wrong. Have gone in a different direction (using a hub instead of the switch) to attempt to solve the original issue.
     
  28. lovingHDTV

    lovingHDTV LI Guru Member

    I installed it and gave it a shot, but I get the following error:

    iptables: No chain/target/match by that name

    I do an iptables -L to list the chains and don't see a PREROUTING or POSTROUTING

    am I missing something?

    thanks,
    dave
     
  29. lovingHDTV

    lovingHDTV LI Guru Member

    Looking earlier in the thread I see that you need to run:

    insmod ipt_ROUTE

    Then it takes the commands.

    I then used:

    iptables -A PREROUTING -t mangle -p udp --dport 10000 -j ROUTE --gw 192.168.1.100 --tee

    This should send a copy of all traffic to my vonage ATA and also a copy to 192.168.1.100. Then on 192.168.1.100 I sniff the SIP packets to get caller information from them.

    What I found happens is that all packets are sent to 192.168.1.100, so the --tee does not appear to be working. I think this is exactly what Wildbill experienced previously.

    Seondly, I found that if you update anything in the web interface it resets the iptables, so you loose everything manually set. Now when things to don't work, and you telephone quits working, then this is a good thing. But when --tee finally does work, I guess I need to find a way to make the "working" rules permanent.

    Any chances of getting the --tee option to work? It would be awesome to have port spanning on these switches.

    thanks,
    dave
     
  30. rhester72

    rhester72 Network Guru Member

    If you put the tee on OUTPUT or POSTROUTING, does it make a difference?

    Rodney
     
  31. lovingHDTV

    lovingHDTV LI Guru Member

    If I use POSTROUTING, then the data gets routed to the gateway, but not copied, so the vonage adapter stops working.

    If I use OUTPUT nothing happens.

    thanks,
    dave
     
  32. rhester72

    rhester72 Network Guru Member

    OK...I can confirm, but not sure what's happening here.

    About the only thing I can recommend is trying *one by one* every chain Tomato offers, including the default (INPUT/OUTPUT/PREROUTING/POSTROUTING) chains that you haven't tried plus the Tomato-specific chains. I'm confident the tee split code is good, but there must be something about the usage we're missing here.

    Rodney
     
  33. lovingHDTV

    lovingHDTV LI Guru Member

    OK I tried all of them:

    iptables -A PREROUTING -t mangle -p udp --dport 10000 -j ROUTE --gw 192.168.1.136 --tee
    iptables -A POSTROUTING -t mangle -p udp --dport 10000 -j ROUTE --gw 192.168.1.136 --tee
    iptables -A INPUT -t mangle -p udp --dport 10000 -j ROUTE --gw 192.168.1.136 --tee
    iptables -A OUTPUT -t mangle -p udp --dport 10000 -j ROUTE --gw 192.168.1.136 --tee

    results:

    PREROUTING: routed to gw, but no phone.
    POSTROUTING: routed to gw, but no phone.
    INPUT: no effect at all
    OUTPUT: no effect at all

    I also tried the wanin chain, but it did not let me to that.
     
  34. rhester72

    rhester72 Network Guru Member

    <patch removed since it was the wrong one :)>

    I'll put up the correct patch (that still apparently isn't working properly) tonight.

    Rodney
     
  35. rhester72

    rhester72 Network Guru Member

    May have a handle on this. More soon.

    Rodney
     
  36. rhester72

    rhester72 Network Guru Member

    Patch management can be a PITA. :)

    Fixed.

    http://multics.dynalias.com/tomato/Tomato_1_23_ROUTE+tee_fixed.7z

    Please give this one a whirl - PREROUTING is the correct chain.

    Correct patch also attached for the developer-minded, as adapted from patch-o-matic for netfilter.

    This tests 100% in my lab - if you have good results I will submit it for consideration for inclusion in mainline Tomato.

    Rodney
     

    Attached Files:

  37. rhester72

    rhester72 Network Guru Member

    Quick update: It would seem you need to use PREROUTING to see outgoing packets (from the source being sniffed) and a second rule against POSTROUTING to see incoming packets (to the source being sniffed). This does, of course, depend on what you are trying to achieve, but the bottom line is that if you want to see both sides of the conversation you'll probably need two rules against two different chains.

    Rodney
     
  38. lovingHDTV

    lovingHDTV LI Guru Member

    All I need is to see the incoming packets to 192.168.1.131. There is no need to see outgoing. But 192.168.1.131 still needs to see the incoming packets as well.

    Does that make sense? That is why I tried it at PREROUTING first.

    What I see is that all the packets get routed away from 192.168.1.131 so it no longer works.
     
  39. rhester72

    rhester72 Network Guru Member

    If you set up your rule against POSTROUTING to destination 192.168.1.131 with the firmware I posted today, it should work fine and split/copy the packets correctly without disruption. I've been testing the hell out of it for half a day and it works as expected.

    Rodney
     
  40. lovingHDTV

    lovingHDTV LI Guru Member

    YESS!!!

    Works perfectly!

    Now to figure out how to make them sticky through a reboot, and the web interface updates. I notice that if I change a port forwarding setting these get wiped out.

    Thanks for fixing this!!
     
  41. rhester72

    rhester72 Network Guru Member

    You should be able to do the insmod in the Init script and the iptables rule in the Firewall script to survive disruption (I _think_). :)

    Rodney
     
  42. lovingHDTV

    lovingHDTV LI Guru Member

    Hehe, just figured that out and it does indeed work!

    dave
     
  43. lovingHDTV

    lovingHDTV LI Guru Member

    I've been using this mod for a while now and everything is still working great.

    Now if we could just get it integrated into the port forwarding GUI :biggrin:
     
  44. puppycrack

    puppycrack Addicted to LI Member

    Awesome job on integrating packet mirroring functionality. I've been looking for this for a long time. :thumbup:

    I have it running, with a POSTROUTING command, and can capture the traffic originating from the ATA to the wan. However, I don't appear to be getting any traffic inboud for the ATA.

    I tried adding a couple different PREROUTING rules in the mangle table, but to no avail.

    Here's my setup (some IP's changed to protect the innocent):
    tomato router (wrt54gs v3): 192.168.1.1
    vonage ATA: 192.168.1.A
    monitoring PC to capture traffic from and to vonage ATA: 192.168.1.B
    * vonage ATA is behind the router. it gets its wan ip from tomato.

    so I created an iptables rule (this captures outbound traffic from ATA only):
    iptables -A POSTROUTING -t mangle -p udp --source 192.168.1.A -j ROUTE --gw 192.168.1.B --tee

    Can anyone let me know what I am missing in order to get traffic coming from WAN to ATA? What format should a PREROUTING rule, if any, take? Any help is greatly appreciated.

    Thanx!

    -pc
     
  45. mstombs

    mstombs Network Guru Member

    Lots of posts in the thread use "iptables -A" this adds rules to the bottom of the filtering chains - I suggest folk also try "iptables -I" to insert them at the top to ensure they are not over-ridden by other rules.
     
  46. puppycrack

    puppycrack Addicted to LI Member

    While I agree with you, and I don't know if your comment was directed towards my question... This does not apply to my situation b/c I had no POSTROUTING rules in place in my mangle table before I attempted to get this set up. So there is no difference between append and insert in my case.
     
  47. mstombs

    mstombs Network Guru Member

    Another cross-fire - why use the "mangle" table - tee is a form of duplicate port forward, port forwarding is done in the "-t nat PREROUTING" and "-t filter FORWARD" chains. Maybe -I and -A more important when using these?
     
  48. lovingHDTV

    lovingHDTV LI Guru Member

    I put my rule in the PREROUTING table as I only want to capture the incoming packets to the ATA.

    I use:
    iptables -A PREROUTING -t mangle -p udp --dport 10000 -j ROUTE --gw 192.168.1.A --tee

    I put this in the firewall script.

    dave

     
  49. msilano

    msilano Network Guru Member

    I'm in the same boat as others here. I've got a Linksys PAP2 that seems to have an issue when connected to my Tomato-based WRT54GS. If a config change stops and restarts some services, the PAP2 seems to lose connectivity...and isn't able to connect to its VOIP gateway. A restart of the PAP2 fixes the issue.

    I was trying to setup an iptables mirror to a server running wireshark, but ran into the same issue as others:

    iptables -A PREROUTING -t mangle --source 192.168.1.194 -j ROUTE --gw 192.168.1.199 -- tee

    > iptables: No chain/target/match by that name

    iptables -A POSTROUTING -t mangle --source 192.168.1.194 -j ROUTE --gw 192.168.1.199 --tee

    > iptables: No chain/target/match by that name

    Now, I know I may be off on my use of the iptables rules, but I still can't seem to get past the matching target/chain.

    Also noted that the iptables version in use is 1.3.7, that is from 2007. Is there any benefit to using a newer (larger, though) version?

    Thanks much,

    Mike
     
  50. rhester72

    rhester72 Network Guru Member

    insmod /lib/modules/2.4.20/kernel/net/ipv4/netfilter/ipt_ROUTE.o
     
  51. msilano

    msilano Network Guru Member

    Brilliant, thanks!

     
  52. i1135t

    i1135t Network Guru Member

    How do you apply the patch to another mod without the patch command? I'd like to test out this -tee option for monitoring website traffic on my LAN.
     
  53. i1135t

    i1135t Network Guru Member

    Ok, just posting the update here as some people may want to know how the port mirroring worked for me and how I got it working as of today.

    The previous comment from rhester72 didn't work for me when I tried to issue this command from SSH.
    Code:
    insmod /lib/modules/2.4.20/kernel/net/ipv4/netfilter/ipt_ROUTE.o
    So I did some searching on the forum and found that SgtPepper posted this a while back and it worked for him here. Then all I did was added "modprobe ipt_ROUTE" into my init script at the top and my iptable changes for PREROUTING chain only in my firewall script and saved & reboot.
     
  54. rhester72

    rhester72 Network Guru Member

    I don't know if SgtPepper's build uses 2.4.20, so the path is probably slightly different.

    The modprobe solution is what I should have suggested, actually, as it is much less sensitive/error-prone. It's a tad slower than insmod, so I use the latter just because I come from an era where shaving cycles meant something, but in practice "modprobe ipt_ROUTE" is simpler and preferred.

    Rodney
     

Share This Page