1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Need som help with small script

Discussion in 'Tomato Firmware' started by kthaddock, Oct 1, 2013.

  1. kthaddock

    kthaddock Network Guru Member

    I'm trying to get this small script working in Tomato, it's working in asuswrt-merlin.
    Script is in: /jffs/kthaddock_peering.sh and look like this (permission 644):
    and with txt-file with this:
    txt-file in: /jffs/ipset_lists/torrent_strings.txt (permission 744)
    When I run this I get this error:
    Best Regards
    kthaddock
     
  2. koitsu

    koitsu Network Guru Member

    Try putting an echo in front of the iptables line to hopefully see what's actually getting run within the for loop.

    I don't particularly see anything wrong, it looks more like iptables isn't able to find the related netfilter/iptables strings module, or is claiming the REJECT target isn't available within the FORWARD chain.

    iptables will often throw wonky/weird errors when not given perfect/exact syntax, so something like $item not getting properly set/expanded might cause something like this, hard to say.
     
  3. kthaddock

    kthaddock Network Guru Member

    Okey thanks koitsu
    Do you mean like this ? echo [ -z "$(iptables-save | grep BitTorrent)"
     
  4. kthaddock

    kthaddock Network Guru Member

    Tried with " echo " infront of iptables and no go.
     
  5. ryzhov_al

    ryzhov_al Networkin' Nut Member

    No. Put echo before iptables command, not iptables-save.
     
  6. kthaddock

    kthaddock Network Guru Member

    Succes !!!!!! but I can't se that when I do iptables -vnL FORWARD
    Maby need reboot first ?
     
  7. koitsu

    koitsu Network Guru Member

    Try entering those iptables commands manually into the CLI, one at a time, to see if they work. If they do, I have some theories/ideas, but will need to dedicate time to figure out the reasoning.
     
  8. kthaddock

    kthaddock Network Guru Member

    I'm using all these command on other router but have them in firewall and they works, block torrent download.
    Maby need to do "nvram commit" after all firewall rules and then reboot.
     
  9. koitsu

    koitsu Network Guru Member

    Note: I am not very happy with the fact that you just now disclose you're doing this on two completely separate routers and on one it works but on the other it doesn't.

    The issue has nothing to do with nvram commit, and now we know it has nothing to do with your script either. Please stop making wild guesses.

    The answer is obvious: there is a difference between the two firmwares you're using. One of them (where things are not working) almost certainly lacks the xt_string.ko netfilter/iptables module, which would absolutely explain the error from iptables ("match" refers to the -m argument (run iptables -h sometime and read!)). You can always test for module availability by doing something like iptables -m string -h and see what comes back. Proper support will show this at the bottom of the output:

    Code:
    STRING match v1.3.8 options:
    --from  Offset to start searching from
    --to  Offset to stop searching
    --algo  Algorithm
    --string [!] string  Match a string in a packet
    --hex-string [!] string  Match a hex string in a packet
    
    In your case, talk to the firmware authors and ask them to verify that the module is being built/included during the kernel build process for the exact firmware you're using (provide the exact filename). A patch is needed as well as the proper kernel configuration file adjustment.

    Respectfully but irked: I am growing very very tired talking about this module. I should have never bothered mentioned it on this forum to begin with. It's a time sink.
     
  10. Victek

    Victek Network Guru Member

    ;) never, it's a knowledge grown .. raf versions:
    Code:
    STRING match v1.3.8 options:
    
    --from                      Offset to start searching from
    --to                        Offset to stop searching
    --algo                        Algorithm
    --string [!] string          Match a string in a packet
    --hex-string [!] string      Match a hex string in a packet
    root@RT-N16:/tmp/home/root# 
     
  11. kthaddock

    kthaddock Network Guru Member

    Thank you taking time to try to help !
    You aske me to test:
    I have this rules on script/firewall on both routers with shibbys 110 and 112-AIO builds all 10 raw and
    that works and verified with iptables -vnL.
    To run them on both router then I know "string" module is in those builds.
    This is a attempt to minimize nvram size and to get god way to start all my rules on my routers.
    I know "string" modul have som lack of functions/switch and I do the best of it with ALL disadvantage.
    I addmit I'm not a programer but I can read, have many times used, iptables -h, string -h and others.

    When I run /jffs/kthaddock_peering.sh those 10 raw get executed but not in FORWARD chain. Isn't needed to restart firewall after that ? "service firewall restart"
    regards
    kthaddock
     
    Last edited: Oct 2, 2013
  12. koitsu

    koitsu Network Guru Member

    Let's recap. When you run the script, you get the following errors:

    Code:
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    iptables: No chain/target/match by that name
    
    To me, this clearly indicates iptables is being run on each line of `cat /jffs/ipset_lists/torrent_strings.txt` (pulled into variable $item), but that the syntax of the iptables line is either wrong or what's being handed to iptables by the underlying script (remember: Busybox is /bin/sh), is something iptables doesn't like. And there is always this possibility: that Busybox's shell is interpreting the script incorrectly (Busybox is a pile of garbage, this wouldn't surprise me).

    I've seen this error happen when things like a matching module (ex. -m foobar) doesn't exist, or when the target (ex. -j foobar) doesn't exist within the particular chain (INPUT vs. FORWARD vs. PREROUTING etc.).

    This is why I asked you if you could simply run the commands by hand in the CLI and if they worked or not. In other words: do the errors happen only happen when you're running the script or do they happen when you enter the relevant iptables commands via CLI? (That is why I asked you to add echo to the front).

    If the problem does not happen from the CLI, but does happen if being run from within Scripts / Firewall or Tools / System, then that's probably a different issue altogether and needs further reverse engineering.

    I can try to reproduce this problem on my own system if needed, but I don't like it when someone says "this works fine on one system but not another".
     
  13. kthaddock

    kthaddock Network Guru Member

    I have run the comand by CLI one by one and its working, can se them in FORWARD chain. (iptables -vnL FORWARD)
    Run "sh /jffs/block_list.sh" both from script/firewall and from putty and doesn't work, can't se rules in FORWARD chain.
    "errors happen only happen when you're running the script" =>YES
    As ryzhov_al says, it's working in asuswrt-merlin builds.
    Maby it's need own chain to work.
     
    Last edited: Oct 4, 2013
  14. koitsu

    koitsu Network Guru Member

    Does the problem happen when using Tools / System to enter the commands?
     
  15. mstombs

    mstombs Network Guru Member

    Check the script has only unix system file endings - could there be an invisible "^M" attached to REJECT?
     
  16. kthaddock

    kthaddock Network Guru Member

    Running from Tools/system:
    With "echo", but isn't in FORWARD chain (iptables -vnL)
     
  17. koitsu

    koitsu Network Guru Member

    Attempting to reproduce using tomato-K26USB-1.28.0503MIPSR2Toastman-RT-N-Ext.trx on an RT-N16 with no "quirky adjustments" made to the CLI environment or anywhere else.

    Commands issued in order:

    1. Insert into the FORWARD chain (as part of the filter table (default)), using the string module (xt_string.ko) which netfilter/iptables knows as string, to block any packets that contain the string "sdfij3i3jsfk1z" (not including quotes; those are for the shell parser), searching within the first 600 bytes of payload, and if found/matched, sent the packet to REJECT chain (built-in chain name within the FORWARD chain).
    2. List off the very first rule in the FORWARD chain.
    3. Delete the rule we just added.

    CLI -- works:

    Code:
    root@gw:/tmp/home/root# iptables -I FORWARD -m string --string "sdfij3i3jsfk1z" --algo bm --from 1 --to 600 -j REJECT
    root@gw:/tmp/home/root# iptables -L FORWARD -n -v --line-numbers | head -3
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    num  pkts bytes target  prot opt in  out  source  destination
    1  0  0 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  STRING match "sdfij3i3jsfk1z" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    root@gw:/tmp/home/root# iptables -D FORWARD 1
    root@gw:/tmp/home/root#
    
    Tools / System -- works:

    Entered into the Command box and clicked Execute:
    Code:
    iptables -I FORWARD -m string --string "sdfij3i3jsfk1z" --algo bm --from 1 --to 600 -j REJECT
    iptables -L FORWARD -n -v --line-numbers | head -3
    
    Output:

    Code:
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    num pkts bytes target prot opt in out source destination
    1 0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 STRING match "sdfij3i3jsfk1z" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    
    I then manually issued iptables -D FORWARD 1 (didn't want to risk breaking things if the original insert did not work).

    Scripted file -- works:

    Code:
    root@gw:/tmp# cat > script.sh
    #!/bin/sh
    iptables -I FORWARD -m string --string "sdfij3i3jsfk1z" --algo bm --from 1 --to 600 -j REJECT
    iptables -L FORWARD -n -v --line-numbers | head -3
    root@gw:/tmp/home/root# cd /tmp
    root@gw:/tmp# chmod 700 test.sh
    root@gw:/tmp# ./test.sh
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    num  pkts bytes target  prot opt in  out  source  destination
    1  0  0 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  STRING match "sdfij3i3jsfk1z" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    
    Again manually issued iptables -D FORWARD 1 to delete the rule I added, then rm ./test.sh.

    Final testing via a script: take a series of lines in a text file, and use them as individual values to the --string parameter for iptables, then once all done, list off the entire contents of the FORWARD chain -- works:

    Code:
    root@gw:/tmp# cat > foo.sh
    #!/bin/sh
    for item in `cat lines.txt`
    do
      iptables -I FORWARD -m string --string "${item}" --algo bm --from 1 --to 600 -j REJECT
    done
    iptables -L FORWARD -n -v --line-numbers
    root@gw:/tmp# chmod 700 foo.sh
    root@gw:/tmp# cat > lines.txt
    dfjuihiuw3hu3uhuwthu
    jksdfksjkjkskjs
    9494hhfhfu82k__93893
    sdfsdjfkds=
    jfjsdjkfkj000
    root@gw:/tmp# ./foo.sh
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    num  pkts bytes target  prot opt in  out  source  destination
    1  0  0 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  STRING match "jfjsdjkfkj000" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    2  0  0 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  STRING match "sdfsdjfkds=" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    3  0  0 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  STRING match "9494hhfhfu82k__93893" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    4  0  0 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  STRING match "jksdfksjkjkskjs" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    5  0  0 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  STRING match "dfjuihiuw3hu3uhuwthu" ALGO name bm FROM 1 TO 600 reject-with icmp-port-unreachable
    6  66  7278 ACCEPT  all  --  br0  br0  0.0.0.0/0  0.0.0.0/0
    7  1334  162K DROP  all  --  *  *  0.0.0.0/0  0.0.0.0/0  state INVALID
    8  136K 8217K TCPMSS  tcp  --  *  *  0.0.0.0/0  0.0.0.0/0  tcp flags:0x06/0x02 TCPMSS clamp to PMTU
    9  18M 6176M ACCEPT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  state RELATED,ESTABLISHED
    10  40271 2869K wanin  all  --  vlan2  *  0.0.0.0/0  0.0.0.0/0
    11  268K  19M wanout  all  --  *  vlan2  0.0.0.0/0  0.0.0.0/0
    12  268K  19M ACCEPT  all  --  br0  *  0.0.0.0/0  0.0.0.0/0
    13  40263 2869K upnp  all  --  vlan2  *  0.0.0.0/0  0.0.0.0/0
    
    Then I deleted all the rules manually, and foo.sh as well as lines.txt.

    Simply put: I cannot reproduce this problem.

    I am fully aware of what you're trying to do in your script. You're saying if the length of the string/results returned from the command iptables-save | grep BitTorrent is zero (meaning there is no match), then proceed to execute a forloop that does the above. However, I do not understand what you're doing with iptables-save (that actually modifies things!). Why are you running that? If you're wanting to look at your existing iptables, you should just look at the file /etc/iptables or just use iptables -L FORWARD -n -v and grep for existing rules/matches.

    I spent nearly 45 minutes today trying to write a replacement script for you that makes sense/works/does what you need. I was unable to accomplish this task due to what are glaringly obvious bugs in the Busybox shell/script parser. I'm sorry. I've written shell for years, but what I'm encountering is flat out a completely broken shell parser in Busybox, thus I will not bother with this crap. What a complete utter piece of garbage. It's [ and test operators, along with equality testing and things like if [ are completely utterly broken. I repeat: BROKEN. I'm basically to the point of telling everyone to just use Entware, install bash, and make sure all their shell scripts use /opt/bin/bash. God Busybox is such garbage...

    Please make sure your torrent_strings.txt file is UNIX format, not DOS format (i.e. LF only, not CR+LF).

    Next, I should also note that the methodology you're using with the string module is incorrect. For TCP packets only you need to use -p tcp ... -j REJECT --reject-with tcp-rst. Your current methodology isn't going to support that. Returning/injecting TCP RST for TCP packets getting blocked is very very important. You can't do this for UDP (you have to rely entirely on ICMP port-unreach), but with TCP it's important. I've talked about this at length multiple times on the forum.

    I cannot help past this point -- I have spent way too much time on this thread as is.
     
  18. kthaddock

    kthaddock Network Guru Member

    Now I have time to teste this again. This is REALLY strange. Have done as Koitsu and it's not working.
    I suspect it's Shibbys build have som trouble with this. (Or I'm doing some strange settings...)
    I'm running my script from JFFS that is only different now.
    Tested from /tmp direct:
    Same setup as koitsu have tested with.
    Yes I have modify my script with Notepad++ in EOL mode to prevent any CR.
    I have noticed your suggestion about "-p tcp ... -j REJECT --reject-with tcp-rst" then I have to use rules for TCP and UDP separate.
     
    Last edited: Oct 8, 2013
  19. Bird333

    Bird333 Network Guru Member

    Just curious but what are you ultimately trying to accomplish with this script?
     
  20. kthaddock

    kthaddock Network Guru Member

    I got It to work, seems that string module doesn't start automatic have to start with " insmod xt_string"
    Have all "item" in a txt file to block torrent download, this generate in my case 20 ruleset i FORWARD chain.
    THANK YOU "Koitsu" for all your help.
     
    Last edited: Oct 10, 2013
  21. shibby20

    shibby20 Network Guru Member

  22. kthaddock

    kthaddock Network Guru Member

    Hehe ;) I noticed that, I know I asked before and then module loaded when needed.
    Just curious which modules do I have to start manually ?

    Best Regards
     
  23. kthaddock

    kthaddock Network Guru Member

    I have used this script for awhile and it's running very god.
    I need to modify as TCP/UDP isn't needed with all torrent strings. Either in two files one for udp and one for tcp.
    (torrent_tcp.txt / torrent_udp.txt) or one file and mark torrent string "torrent/ut - torrent/t - torrent/u"
    How can I modify script to get this?:
    regards
    kthaddock
     
  24. ryzhov_al

    ryzhov_al Networkin' Nut Member

    File with patterns:
    Code:
    # cat ./torrent_strings.txt
    udp;tracker
    udp;d1:ad2
    tcp;torrent
    tcp;info_hash
    
    The script itself:
    Code:
    #!/bin/sh
    
    insmod xt_string
    for item in $(cat /opt/tmp/torrent_strings.txt)
    do
       proto=$(echo $item | awk -F";" "{ print \$1 }")
       string=$(echo $item | awk -F";" "{ print \$2 }")
       [ "$proto" = "udp" ] && iptables -I FORWARD -p udp -m string --string "${string}" --algo bm --from 1 --to 600 -j REJECT
       [ "$proto" = "tcp" ] && iptables -I FORWARD -p tcp -m string --string "${string}" --algo bm --from 1 --to 600 -j REJECT
    done
     
  25. kthaddock

    kthaddock Network Guru Member

    Thank you ryzhov_al !
    I will test tonight or tomorrow.

    Best Regards
    kthaddock
     
  26. kthaddock

    kthaddock Network Guru Member

    Working just fine, have tested.
    Only thing is when I run script second time it's compalin "xt_string.ko" File exists
     
  27. ryzhov_al

    ryzhov_al Networkin' Nut Member

    It's just a warning. No need to load kernel module if it's already loaded:
    Code:
    [ -z "$(lsmod | grep ^xt_string)" ] && insmod xt_string
     
  28. kthaddock

    kthaddock Network Guru Member

    Thank you !
    Now it's working and no complain from module.

     
  29. kthaddock

    kthaddock Network Guru Member

    This script above this post working very well, but i'm now trying to modified that.
    Can someone take a look and see if i have done anything stupid :eek:

     
    Last edited: Feb 25, 2014
  30. koitsu

    koitsu Network Guru Member

    Code:
    if [ -z "$(iptables-save | grep torrent_strings)" ]
    do
    ...
    done
    
    This looks wrong; do should be then, and done should be fi.
     
  31. leandroong

    leandroong Addicted to LI Member

  32. mstombs

    mstombs Network Guru Member

    On the subject of scripting style don't most firmwares have modprobe - which means

    Code:
    [ -z "$(lsmod | grep ^xt_string)" ] && insmod xt_string
    can be replaced with

    Code:
    modprobe xt_string
    and can use built-in string tools rather than echo/awk to replace

    Code:
    proto=$(echo $item | awk -F";" "{ print \$1 }")
    string=$(echo $item | awk -F";" "{ print \$2 }")
    with

    Code:
    proto=${item%;*}
    string=${item#*;}
    or you could read the contents of the file line by line straight into the variables using IFS and read, for example

    Code:
    IFS=";"
    while read proto string; do echo $proto $string; done <torrent_strings.txt
     
    Last edited: Feb 26, 2014
  33. kthaddock

    kthaddock Network Guru Member

    koitsu, leandroong, mstombs
    Thank you for your suggestions, I gone try later on this week and see if it working.


    kthaddock :rolleyes:

     
  34. kthaddock

    kthaddock Network Guru Member

    I have now tested and have some problem with bad argument.
    Trying to get this working and then test mstombs suggestion.

    -------------
    ---------
    Torrent string:
     
    Last edited: Mar 11, 2014

Share This Page