1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

P2Partisan 3.01 - mass IP blocking - peerblock/peerguardian for tomato

Discussion in 'Tomato Firmware' started by rs232, Oct 11, 2013.

  1. rs232

    rs232 LI Guru Member

    This script is designed to protect your tomato router and your LAN client from communicating with unwanted IP inbound and outbound. It is an equivalent of the peerblock/peerguardian software for windows and meant to be a single point of administration for the LAN. The lists can easily be found in Internet but one of the main providers is iblock (www.iblocklist.com). All the credit to RMerlin who wrote the original peerguardian script

    I`ll call this P2Partisan

    What does it exactly do? Here the description of peerblock, the same applies here:
    P2Partisan lets you control who your computer "talks to" on the Internet. By selecting appropriate lists of "known bad" computers, you can block communication with advertising or spyware oriented servers, computers monitoring your p2p activities, computers which have been "hacked", even entire countries! They can't get in to your computer, and your computer won't try to send them anything either

    ScreenShot050.png

    P2Partisan follows the following logic:
    a) block any communication with IPs listed in blacklist-custom
    b) allow traffic from/to ports whitelisted no matter what the IP is
    c) block IPs as per provider based lists
    d) allow everything else

    Few additional considerations:
    - The script uses ipset which is not included in all the mods out there. It was developed on Shibby's 2.6 builds and thought to be working on RMerlin builds too. If unsure try to run ipset from the command line first. e.g. on shibby 2.4 builds ipset is not available.
    - it supports multiple lists via a single "blacklists" file in the format [name url]
    - it supports a single "whitelist" file in the format of IP range [X.X.X.X-Y.Y.Y.Y]
    - Since outbound protection has been implemented the white list must not contain private IP addresses.
    - P2Partisan allows to specify white listed port (ports you would never want to block no matter what IP they are coming from)
    - white listed ports can be configured editing the beginning script. So you'll want to do this if you host whatever internal server (web/mail/pop3/imap/openvpn/else)
    - packet dropped are logged into the system log. The default allows 6 entry every hour and this is already enough to flood your log and not see anything else. Consider reducing this value to 1 or 2 once the script is stable. BTW 6 in this case does not mean 6. Ref: http://www.oocities.org/youssef116/writing/ratelim.html
    - Changed a lot during your tests? Consider restarting your device
    - The right way to run P2Partisan is from Administration/Script/Firewall, otherwise the iptables rules might disappear when other services restart the firewall. However: no reason to fear P2Partisan can set itself in autorun mode (read below)
    - Be considerate when enabling multiple black lists as RAM can be an issue. The default blacklists should work fine on devices with 64MB of RAM
    - it is advised to run this script from some sort of external media (cifs/usb)

    <CHANGELOG>
    p2partisan v3.01
    - split P2PARTISAN-DROP chain into two: P2PARTISAN-DROP-IN/P2PARTISAN-DROP-OUT
    - split DROP is used for the WAN and REJECT for the LAN
    - reject is now specifying --reject-with icmp-admin-prohibited
    - changed the status command to reflect the change
    - White ports have no limit of 15 entries any more
    - added extra rule to process outbound LAN traffic as soon as it hits br0
    - speed improvement

    p2partisan v2.51
    - changed default action from 'DROP' to 'REJECT --reject-with icmp-proto-unreachable'
    - added support for custom black IPs via blacklist-custom file
    - optimizations

    p2partisan v2.40
    - minor improvements
    - bugfixes
    - resolve the stuck on "Loading..." scenario

    Older entries removed
    </CHANGELOG>


    <INSTALLATION last edited 28/08/2014>

    Upgrade (if you don't want to run a fresh install):
    Run the following code within the existing p2partisan directory and then reboot
    NOTE: This will intentionally preserve the existing IP blacklist, blacklist-custom and IP whitelist files however whiteports are overwritten as they are currently defined within the p2partisan.sh file.

    Code:
    # Run within the existing p2partisan directory! #######
    PWD=`pwd`
    rm *.sh
    rm *.gz
    rm *-add
    rm *-del
    wget http://pastebin.com/raw.php?i=bPHBa1tW -O p2partisan.sh
    sed "s#/cifs1/p2partisan#$PWD#g" -i ./p2partisan.sh
    tr -d "\r"< ./p2partisan.sh > ./.temp ; mv ./.temp ./p2partisan.sh
    chmod -R 777 ./p2partisan.sh
    ./p2partisan.sh autorun-off
    ./p2partisan.sh autoupdate-off
    

    New installation:
    Change the INSTALLDIR variable only (second line), of the following script (I'll use /cifs1/ in this example) and run the code within a ssh session or via the GUI - tools/system commands:
    Code:
    #Where should I create the p2partisan directory?
    INSTALLDIR=/cifs1
    #End of configuration ########
    cd $INSTALLDIR
    rm -fR p2partisan
    mkdir p2partisan
    cd p2partisan
    PWD=`pwd`
    # get the script
    wget http://pastebin.com/raw.php?i=bPHBa1tW -O p2partisan.sh
    sed "s#/cifs1/p2partisan#$PWD#g" -i ./p2partisan.sh
    tr -d "\r"< ./p2partisan.sh > ./.temp ; mv ./.temp ./p2partisan.sh
    # get the blacklists
    wget http://pastebin.com/raw.php?i=ARx7NAYz -O blacklists
    tr -d "\r"< ./blacklists > ./.temp ; mv ./.temp ./blacklists
    # get the blacklist-custom
    wget http://pastebin.com/raw.php?i=2xkwzR1A -O blacklist-custom
    tr -d "\r"< ./blacklist-custom > ./.temp ; mv ./.temp ./blacklist-custom
    # get the whitelists
    wget http://pastebin.com/raw.php?i=eb0V3YLp -O whitelist
    tr -d "\r"< ./whitelist > ./.temp ; mv ./.temp ./whitelist
    chmod -R 777 $INSTALLDIR/p2partisan
    
    - Line 7 in p2partisan.sh should be automatically adjusted to your custom path but double check for your peace of mind
    - You might want to have a look at the other parameters set at the beginning of the file just to make sure you're happy with them
    - Edit the blacklists file if needed
    - Edit the blacklist-custom if needed
    - Edit the whitelist file if needed

    </INSTALLATION>


    <EXECUTION last edited 12/05/2014>

    Now you have the initial scripts ready to go.

    Code:
    root@tomato:/cifs1/p2partisan# ls -la
    drwxrwxrwx  2 root  root  100 Nov 22 13:44 .
    drwxrwxrwx  9 root  root  260 Nov 22 13:44 ..
    -rwxrwxrwx  1 110  nas  161 Aug 18 23:05  blacklist-custom
    -rwxrwxrwx  1 110  nas  1300 Aug 18 22:48  blacklists
    -rwxrwxrwx  1 root  root  19 Aug 18 23:19  p2partisan.sh
    -rwxrwxrwx  1 110  nas  248 May 29 23:12  whitelist
    
    From version 1.08 the script accept parameters of whose the default is start.
    p2prtisan.sh help is your friend.

    ScreenShot048.png

    First time execution? After you have double-checked the options at the beginning of the file:

    Run
    Code:
    ./p2partisan.sh
    NOTE: Running the script manually you'll notice that this takes rather long time, especially if fastroutine is disabled!
    NOTE2: Opposite if fastroutine is enabled (default) it is absolutely normal to have the router stuck for few seconds/minutes (dependent on device speed and number of lists)

    That's it. To verify:
    Code:
    ./p2partisan.sh status
    Autorun
    To enable/disable* (*default) autorun:
    Code:
    ./p2partisan.sh autorun-on|autorun-off
    Periodic updates
    To enable/disable* (*default) autoupdate:
    Code:
    ./p2partisan.sh autoupdate-on|autoupdate-off
    By default updates are run every Monday at 4:30am with the paranoia-update parameter (safest). To change the schedule, edit the configuration part for the file and set autoupdate to off then back on again.

    To verify as usual just use status:
    Code:
    ./p2partisan.sh status
    To verify the autoupdate manually:
    Code:
    cru l | grep P2Partisan-update
    </EXECUTION>


    <TROUBLESHOOTING last edited 12/05/2014>

    How to verify if it works or not?

    1)
    Code:
    ./p2partisan.sh status
    ScreenShot051.png

    2) if you have peerguardian/peerblock on your client AND this script running you should see in peerguardian/peerblock only outbound packets (nothing blocked inbound), supposed you have the very same lists set on router and peerblock client. Also note that those connections outbound would never make it as far as the WAN interface.
    3) If you don't have the software use a sniffer e.g. Wireshark filtering the port used by your p2p client
    4) If you have the option syslog=1 set (available from v1.07 and above): check what tomato has dropped via either GUI (Status/Overview/Logs) or ssh (tail /var/log/messages) -->
    5) run the commands:
    iptables -nvL INPUT | grep P2PARTISAN
    iptables -nvL OUTPUT | grep P2PARTISAN
    iptables -nvL wanin | grep P2PARTISAN
    iptables -nvL wanout | grep P2PARTISAN

    and note the packet count (first column). e.g.
    Code:
     pkts bytes target     prot opt in     out     source               destination
    3950  396K P2PARTISAN-IN  all  --  any    any     anywhere             anywhere            state NEW
    
    in this case 3950 packets matched in INPUT with status NEW (no existing connection) are spotted and directed to the P2PARTISAN-IN table

    NOTE: iptable counters are updated periodically, it's not real time! If you see no traffic... wait and re run after e.g. 30 seconds

    iptables -nvL P2PARTISAN-IN
    iptables -nvL P2PARTISAN-OUT

    and now (still in the first column) the number of packet matched by white/blacklist. The last line identifies what is matched by the blacklist
    Code:
    Chain P2PARTISAN-IN (1 references)
    pkts bytes target     prot opt in     out     source               destination
        0  0 P2PARTISAN-DROP-IN  all  --  *  *  0.0.0.0/0  0.0.0.0/0  match-set blacklist-custom src
        2   296 ACCEPT     tcp  --  any    any     anywhere             anywhere            multiport sports www,https,imaps,smtp,ftp
        1    95 ACCEPT     udp  --  any    any     anywhere             anywhere            multiport sports www,https,993,25,fsp
       38  1976 ACCEPT     tcp  --  any    any     anywhere             anywhere            multiport dports www,https,imaps,smtp,ftp
        0     0 ACCEPT     udp  --  any    any     anywhere             anywhere            multiport dports www,https,993,25,fsp
        0     0 ACCEPT     all  --  any    any     anywhere             anywhere            set whitelist src,dst
       15  1774 P2PARTISAN-DROP-IN  all  --  any    any     anywhere             anywhere            set level1 src,dst
    
    15 packets are NEW connections in INPUT matching the blacklist IP range. These packets are now sent to the P2PARTISAN-DROP table to be logged and dropped.

    Finally check the drops:
    iptables -nvL P2PARTISAN-DROP-IN
    iptables -nvL P2PARTISAN-DROP-OUT

    Packet counted here means logged and dropped. It is normal to have a different number due to the maximum log/min option specified in the p2partisan script itself. It is also normal not to have the LOG action if you have specified so in the options of the p2partisan.sh
    Code:
    Chain P2PARTISAN-DROP-IN (10 references)
    pkts bytes target  prot opt in  out  source  destination
      1  129 LOG  all  --  *  *  0.0.0.0/0  0.0.0.0/0  limit: avg 1/hour burst 1 LOG flags 0 level 1 prefix "P2Partisan Rejected: "
      15  2200 REJECT  all  --  *  *  0.0.0.0/0  0.0.0.0/0  reject-with icmp-admin-prohibited
    
    15 Packed actively rejected as they matched either list based or custom blacklist.

    </TROUBLESHOOTING>


    <UNINSTALL>

    Code:
    ./p2partisan.sh autorun-off
    ./p2partisan.sh autoupdate-off
    ./p2partisan.sh stop
    rm -R /whereveryouhaveinstalledit/p2partisan
    
    </UNINSTALL>

    If you spot something that can be added/removed/improved please do let me know!

    P.S. Sorry to Aruba islanders to have used their constituent country as an example in the blacklist country section :rolleyes:

    P.P.S. Writing and testing this script I realised how long Internet torrent clients keep hammering your torrent port asking for connection literally hours after you have close the program. Try it yourself!

    Thanks
    rs232
     
    Last edited: Aug 28, 2014
    Goggy likes this.
  2. jerrm

    jerrm Addicted to LI Member

    Nice, a question and a couple of suggestions/comments:

    Just curious, what router are you using? The level1/level2 lists are huge and will likely exhaust a 64MB ram router, almost certainly 32MB. Might work with swap enabled. I'll try to test.

    Again, with regard to memory, I've found nethash/iphash to be more memory efficient than iptreemap. An ipset hash can only have 64K elements though, so you'd have to break the larger lists into multiple sets inside a setlist. Probably not worth it for you, but may help others.

    Speed will likely be much improved by formatting the output so an ipset "restore" can be done, vs calling ipset for each line.

    A rule isn't necessarily needed for the whitelist unless you want to avoid the ipset check. You could just delete the ranges from the blacklist ipsets. One advantage of iptreemap vs nethash is that if 10.0.0.0-10.255.255.255 is added, you can then delete just the subset range like 10.10.10.0-10.10.10.255 or a single IP.

    Lastly and maybe most importantly, the script is blindly inserting the rules at position 2,3,5,13 of the forward table. 2,3,5 will be before the state rule in many instances, causing a lot of unnecessary traffic to be tested. 13 could be afer some other accept rule. Your "default" rule positions are almost certainly different than mine. To generalize the script, I'd either test for the position of the state rule, or use the wanin/wanout chains to make it easy (it's why they exist). Also, LAN addresses may not need to be whitelisted if using wanin/wanout.
     
  3. jerrm

    jerrm Addicted to LI Member

    Did a little testing using the level1 list on a 64M unit.

    First:
    Memory usage is better than I expected based on my prior testing, but we may still be pushing the limit for a 64M router(see second point). Iptreemap memory usage is definitely not linear based on the number of ranges.

    I think part of it is my nethash vs iptreemap testing was with country netblock lists of around 20K lines. These were mostly very large blocks. The level1 list is 10x the size of what I was testing by line count, but the ranges are much smaller, with many single IPs.

    Second:
    Using ipset -R is MUCH faster than line by line - by orders of magnitude - but only IF it works. Very hit and miss, often the restore is tuncated. Could be memory related, could be a bug in the code. I will test later on an N66U to see if the restore has the same hit or miss behavior with more available memory. Even if additional memory fixes the problem, the only safe option would be line by line for a generic script. Could try splitting it into restores of maybe 20K lines each. Might improve speed and not hit memory barriers. More testing if I get bored.
     
    Last edited: Oct 12, 2013
  4. jerrm

    jerrm Addicted to LI Member

    Using an ipset restore seems solid on the N66U, so it must be a memory issue for the 64MB unit. It's a shame, because it cuts the load time of the level1 list from about 19 minutes to 24 seconds on the N66U. Probably would be worth the trouble to test breaking the file into smaller chunks.

    If you want to test, replace:
    Code:
    #Load the blacklists
    if [ "$(ipset --swap $name $name 2>&1 | grep 'Unknown set')" != "" ]
      then
      ipset --create $name iptreemap
      [ -e $name.lst ] || wget -q -O - "$url" | gunzip | cut -d: -f2 | grep -E "^[-0-9.]+$" > $name.lst
      for IP in $(cat $name.lst)
      do
      ipset -A $name $IP
      done
    fi
    with:
    Code:
    #Load the blacklists
    if [ "$(ipset --swap $name $name 2>&1 | grep 'Unknown set')" != "" ]
      then
      [ -e $name.lst ] || wget -q -O $name.lst "$url"
      { echo "-N $name iptreemap"
        gunzip -c  $name.lst | \
        sed -e "/^[\t ]*#.*\|^[\t ]*$/d;s/^.*:/-A $name /"
        echo COMMIT
      } | ipset -R
    fi
     
    Last edited: Oct 12, 2013
  5. rs232

    rs232 LI Guru Member

    Thanks jerrm! I really appreciate your inputs. I'll have a look into everything you're written and see what it could me modified/improved.

    Edited:
    I've developed this on an ASUS RT-N16, I've enabled
    level1
    level2
    spywere
    spiders
    hijacked

    and have 57% of free RAM. To be said my router is doing quite few other things. I think the main different compared to the peerguardian script is the fact that I'm not using /tmp so no RAM used for pure storage.

    Interesting point, I suppose it depends on how many lists you're loading. Multiple ipset (other than one per list) might become difficult to manage though, considering I was thinking to implement a remove/restore function to destroy the lists not needed without necessarily having to reboot.
    On the memory side: Level1 is definitely a must have where I consider the others nice to have personally. I haven't tried yet but level1 would not run on a 4MB device, and I suspect not even on a 8MB. This makes the ipblock functionality in general something for high end devices 32MB or more. On this point I'm going to try this tonight on my netgear 3500L that comes with 8MB. I doubt though...

    Not sure I get this point, can you please elaborate? I think any consideration that would improve optimise should be taken into account! I'm interested

    I see where you're coming from, so basically act after the ipset loads the list. I'll have a look into this as it would save the extra iptables ALLOW line

    That's correct I've picked up positions based on beginning(ish) end(ish) of the lists specifically because rules change and positions are not absolute. I like the wanin/out idea well spotted! I think I'm going to change this

    rs232
     
    Last edited: Oct 14, 2013
  6. rs232

    rs232 LI Guru Member

    About the whitelist:

    I've just figured out that that would work in case you want to whitelist the whole ip range (some ranges have thousands of IP). So yes you could remove them but I found the iptables more "precise"

    About the nethash, the lists provided by iblock are actually in iptreemap format: source IP to end IP. Where nethash needs: network and mask.
    I don't see how nethash can work unless writing lots of code

    Unless I'm missing something quite simple instead
     
    Last edited: Oct 14, 2013
  7. jerrm

    jerrm Addicted to LI Member

    128MB RAM is probably plenty of headroom.

    The Level1 list is borderline for a 64MB router. It loads reliably using line-by-line "ipset -A" commands but when using restore (ipset -R or ipset --restore), the whole list may not load. Apparently there is more overhead needed for the --restore option.

    Using --restore for the Level1 list seems OK on a "clean" 64MB unit, but add in anything that eats 4MB or so of RAM and it gets dodgy. Having swap enabled doesn't help.

    The speed difference between line-by-line and ipset --restore is 20 minutes vs 20 seconds. I would use --restore and just post a disclaimer the script needs 128MB+ for the Level1 list. None of the other lists are big enough to be an issue.

    I tested using nethash/iphash sets in a setlist for Level1 list, and memory usage was pretty close to iptreemap. The nethash/iphash setlists were maybe just a little smaller, but no where near enough to justify the extra time required for the range to CIDR translation. Probably should post my deaggregate script and see if anyone can speed up it up.

    See post #4 above.

    Avoiding the ipset check is not necessarily a bad idea, but if trying to write a generic script the rule could also prevent some other rule from executing that may need to run.
     
  8. rs232

    rs232 LI Guru Member

    Implemented... Just one thing though, when I run I get a:

    root@tomato:/cifs1/p2partisan# ./p2partisan.sh
    loading modules
    loading ports 80,443 exemption
    loading the whitelist
    Setting whitelist iptables
    loading blacklist 1 - level1
    gunzip: invalid magic
    Setting FORWARD iptables
     
    Last edited: Oct 14, 2013
  9. jerrm

    jerrm Addicted to LI Member

    Did you delete the $name.lst files?

    If you took my example as-is it is expecting name.lst to be the gzipped file, I probably should have changed it to $name.lst.gz.
     
    rs232 likes this.
  10. rs232

    rs232 LI Guru Member

    that was actually the problem thanks!

    I've noticed that your routing is both very fast and resource intensive. So I added an option called fastroutine which is set by default but the user can decide whether to enable it or not.

    I also changed you routing to stick to .gz files to avoid confusion and problems in case somebody likes to switch from one routine to the other.

    It's all packed up into v1.04 (see first post)

    Many thanks for the input, I'm going to look into the iptables rule positioning next
     
  11. Goggy

    Goggy LI Guru Member

    Hi!

    First of all thx for this script! Shouldnd be the generated "iptables-add" executable?
    ./p2partisan.sh: line 149: iptables-add: Permission denied
     
  12. rs232

    rs232 LI Guru Member

    Hi!

    It does on my router! Try to remove the files manually, re-run the p2partisan.sh and see what is the default permission. On my router its 744 (rwxr--r--). Also what media are you using?

    Thanks!

    Edited: try to re-download the script, I've added the relevant chmod within the script.
     
    Last edited: Oct 14, 2013
    Goggy likes this.
  13. Goggy

    Goggy LI Guru Member

    Hi!

    I'm using your script on a "shibby-powered" RT-AC66U. As external storage i use a usb-stick.
    I redownloaded your script, deleted the "." in the chmod - command (chmod .777 ./iptables-* to chmod 777 ./iptables-*) and now its executing as expected. :)
    After that i activated every "generic list" in your blacklist. Script executed fine again and is using about 40 MB of ram now.

    Thank you for this nice little thing :)
     
    darkknight93 likes this.
  14. 68rustang

    68rustang Network Newbie Member

    I was messing around with this over the weekend a little bit on my WNR3500L V2. I followed the directions in the first post. The only change I made was instead of using "/cifs1/p2partisan/p2partisan.sh" I am trying to run it from the same USB I am using for Adblocking, "/tmp/mnt/USB/p2partisan/p2partisan.sh"

    I tried to set it to run automatically (because I do not know any other way) and nothing seems to happen.

    In the WANUP section I have:

    /tmp/mnt/USB/adblock/adblock.sh
    Sleep 180
    /tmp/mnt/USB/p2partisan/p2partisan.sh

    Rebooting the router and watching the logs I don't see anything after the adblocking does its thing. I have tried changing the sleep interval but that doesn't seem to do anything.

    Is there A peerblock script for dummies section yet?
     
  15. rs232

    rs232 LI Guru Member

    Sorry from your post is not clear whether you have modified the 7th line of p2partisan or not. Just in case you need to modify the path within the script. So in your case change
    Code:
    cd /cifs1/p2partisan
    into
    Code:
    cd /tmp/mnt/USB/p2partisan
    Not sure what you mean... Just run the scrip as told and you will not need anything else.

    HTH
    rs232
     
  16. 68rustang

    68rustang Network Newbie Member

    The peerblock for dummies question was a joke :) Though I obviously need it since I am having trouble getting something so simple to run...

    Anyway, I went back and double checked the 7th line of p2partisan.sh and made sure it was
    Code:
    cd /tmp/mnt/USB/p2partisan
    it still doesn't work.

    If I paste:
    Code:
    /tmp/mnt/USB/p2partisan/p2partisan.sh
    Into TOOLS>SYSTEM COMMANDS

    I get:
    Code:
    /tmp/.wxumzUrD: line 5: /tmp/mnt/USB/p2partisan/p2partisan.sh: not found
    I can browse to it through the network, I can see it, it is there. All filenames are correct. All three files are there.

    The USB is mounted and is shown under USB and NAS as:
    Code:
    Partition 'USB' vfat (7,617.50 MB / 7,616.36 MB free) is mounted on /tmp/mnt/USB
    I know the router should be able to see the USB because adblocking is running from a folder on the same drive without issues.

    I do not see anything in the logs.

    I have to be missing something simple...

    EDIT:

    Thinking about this some more when I was able to setup adblocking the *.sh files were actually installed using another "script" that was pasted into the COMMANDS windows and run. With this I am creating the p2partisan.sh file directly and placing it in the directory. Could this have anything to do with my issues?
     
    Last edited: Nov 19, 2013
  17. rs232

    rs232 LI Guru Member

    Try:

    Code:
    chmod -R 777 /tmp/mnt/USB/p2partisan
    and re-run

    P.S. is the USB mounted read-write? It must be to make this working
     
  18. 68rustang

    68rustang Network Newbie Member

    still getting
    Code:
    /tmp/.wxbmWbO2: line 7: /tmp/mnt/USB/p2partisan/p2partisan.sh: not found
    /USB is mounted READ/WRITE but just to be safe I explicitly shared ../USB/p2partisan as READ/WRITE as well. Still getting the "not found" error.

    I redid the p2partisan.sh file using NotePad++ instead of Notepad, same error.
     
  19. rs232

    rs232 LI Guru Member

    How about if you run it like this?

    Code:
    cd /tmp/mnt/USB/p2partisan/ ; ./p2partisan.sh
    
    For reference can you post what you have in line 7 of the script?

    Regards
     
  20. 68rustang

    68rustang Network Newbie Member

    Pasting
    Code:
    cd /tmp/mnt/USB/p2partisan/ ; ./p2partisan.sh
    into the COMMAND windows returns:
    Code:
     /tmp/.wxaxDc67: line 5: ./p2partisan.sh: not found  
    Line 7 of p2partisan.sh is
    Code:
     cd /tmp/mnt/USB/p2partisan
    changing it to
    Code:
    cd /tmp/mnt/USB/p2partisan/ ; ./p2partisan.sh
    same thing
    I notice it is now saying line5 whereas before it ws referencing Line 7.
     
  21. rs232

    rs232 LI Guru Member

    I'm not sure what/were it went wrong but as other users are using it I suspect there's something strange that happened on your side.

    Might I suggest you run the following code from ssh?

    Code:
    cd /tmp/mnt/USB/
    rm -fR p2partisan
    mkdir p2partisan
    cd p2partisan
    wget http://pastebin.com/raw.php?i=zMthLn2R -O p2partisan.sh
    wget http://pastebin.com/raw.php?i=ARx7NAYz -O blacklists
    wget http://pastebin.com/raw.php?i=eb0V3YLp -O whitelist
    chmod -R 777 /tmp/mnt/USB/p2partisan
    ./p2partisan.sh
     
  22. rs232

    rs232 LI Guru Member

    I've modified the original post to simplify the installation.

    HTH
    rs232
     
    Last edited: Nov 21, 2013
  23. 68rustang

    68rustang Network Newbie Member

    Same error running the above code...

    I really do appreciate you helping me with this, Thank you.

    The above code didn't work so I tried the edited installation routine.

    Code:
    #Where should I create the p2partisan directory?
    INSTALLDIR="/cifs1"
    cd $INSTALLDIR
    rm -fR p2partisan
    mkdir p2partisan
    cd p2partisan
    PWD=`pwd`
    # get the script
    wget http://pastebin.com/raw.php?i=zMthLn2R -O p2partisan.sh
    sed "s#/cifs1/p2partisan#$PWD#g" -i ./p2partisan.sh
    # get the blacklists
    get http://pastebin.com/raw.php?i=ARx7NAYz -O blacklists
    # get the whitelists
    wget http://pastebin.com/raw.php?i=eb0V3YLp -O whitelist
    chmod -R 777 $INSTALLDIR/p2partisan
    
    I changed INSTALLDIR="/cifs1" to INSTALLDIR="/tmp/mnt/USB/p2partisan"

    It also doesn't work for me. It creates the directories and the p2partisan.sh and WHITELIST files but it does not automatically change the install dir in p2partisan.sh or get the BLACKLIST file. I manually edited the p2partisan.sh file and tried to rerun the script but get the same not found error.

    Code:
    sed "s#/cifs1/p2partisan#$PWD#g" -i ./p2partisan.sh
    doesn't look right to me. shouldn't it be either:
    Code:
    sed "s#/$INSTALLDIR/p2partisan#$PWD#g" -i ./p2partisan.sh
    or
    Code:
    sed "s#/tmp/mnt/USB/p2partisan#$PWD#g" -i ./p2partisan.sh
    This is what I see in PuTTY

    Code:
    login as: root
    root@192.168.2.1's password:
     
    Tomato v1.28.0000 MIPSR2-114 K26 USB AIO
    ========================================================
    Welcome to the Netgear WNR3500L v2 [router]
    Uptime:  15:02:52 up 5 days, 16:34
    Load average: 0.00, 0.00, 0.00
    Mem usage: 14.5% (used 17.98 of 123.77 MB)
    WAN : xxx.xxx.xxx.xxx/22 @ xx:xx:xx:xx:xx:xx
    LAN : 192.168.2.1/24 @ DHCP: 192.168.2.100 - 192.168.2.149
    WL0 : Home @ channel: 6 @ xx:xx:xx:xx:xx:xx
    ========================================================
    root@router:/tmp/home/root# #Where should I create the p2partisan directory?
    root@router:/tmp/home/root# INSTALLDIR="/tmp/mnt/USB"
    root@router:/tmp/home/root# cd $INSTALLDIR
    root@router:/tmp/mnt/USB# rm -fR p2partisan
    root@router:/tmp/mnt/USB# mkdir p2partisan
    root@router:/tmp/mnt/USB# cd p2partisan
    root@router:/tmp/mnt/USB/p2partisan# PWD=`pwd`
    wget http://pastebin.com/raw.php?i=zMthLn2R -O p2partisan.sh
    root@router:/tmp/mnt/USB/p2partisan# # get the script
    root@router:/tmp/mnt/USB/p2partisan# wget http://pastebin.com/raw.php?i=zMthLn2R
    -O p2partisan.sh
    # get the blacklists
    get http://pastebin.com/raw.php?i=ARx7NAYz -O blacklists
    # get the whitelists
    wget http://pastebin.com/raw.php?i=eb0V3YLp -O whitelist
    chmod -R 777 $INSTALLDIR/p2partisanConnecting to pastebin.com (190.93.243.15:80)
    p2partisan.sh  100% |*******************************|  4924  0:00:00 ETA
    root@router:/tmp/mnt/USB/p2partisan# sed "s#/$INSTALLDIR/p2partisan#$PWD#g" -i .
    /p2partisan.sh
    root@router:/tmp/mnt/USB/p2partisan# # get the blacklists
    root@router:/tmp/mnt/USB/p2partisan# get http://pastebin.com/raw.php?i=ARx7NAYz
    -O blacklists
    -sh: get: not found
    root@router:/tmp/mnt/USB/p2partisan# # get the whitelists
    root@router:/tmp/mnt/USB/p2partisan# wget http://pastebin.com/raw.php?i=eb0V3YLp
    -O whitelist
    Connecting to pastebin.com (190.93.242.15:80)
    whitelist  100% |*******************************|  259  0:00:00 ETA
    root@router:/tmp/mnt/USB/p2partisan# chmod -R 777 $INSTALLDIR/p2partisan
    
    I then manually edit line#7 from cd /cifs1/p2partisan to cd /tmp/mnt/USB/p2partisan. Then I paste /tmp/mnt/USB/p2partisan/p2partisan.sh into the COMMAND window and click EXECUTE and get the same line 5 error. Why is it saying line 5? Line 5 is a bunch of "#". If I edit out Line 5 it still complains about line 5.

    Normally at this point I would assume I am in over my head and give up but this seems so simple and it is functionality I have been wanting on the router for a looooooong time.

    EDIT: I also tried disabling fastroutine just to see if that had anything to do with it. It didn't make a difference.
     
  24. rs232

    rs232 LI Guru Member

    Hi again, no problems, let's see if we can sort this out!

    I'm trying to get my head around what it might have happened. Things I can think about are:

    - the installation script (as per post 1) had a typo with the black list (get instead of wget) this is now solved
    - The sed line does work fine (just tried) as the default file (pastebin link) has /cifs1/p2partisan in it as default path. The sed will replace it with the actual dir on your system. I would re-run the script and see what happens.
    - The line 5 error doesn't look good to me, you're right line 5 is a list of "#". Nobody has ever complained about this before so I guess it's and char-set problem? Can you try to remove all the comments (lines starting with a "#"?
    - what editor do you use? I would stick to shell based on like pico/nano/vi

    Just one final note,
    as the script creates the p2partisan, you should use instead:
    Code:
    INSTALLDIR="/tmp/mnt/USB"
     
    Last edited: Nov 22, 2013
  25. 68rustang

    68rustang Network Newbie Member

    blacklist is now downloading with the others. :cool:
    I understand the sed line better now and it is changing the directory to the proper "/tmp/mnt/USB" in p2partisan.sh :cool:
    Still getting the line 5 error, same as before, even after removing all the comments :confused:
    I am using Notepad++ If you have any suggestions for Windows I will try them.

    I was, I typed it wrong in the previous post.

    Just to eliminate any weirdness I disabled adblock.sh and tried again, still line 5 not found error.

    Is that error saying it cannot find p2partisan.sh or it cannot find something on line 5 of the p2partisan.sh script?

    I also updated the router to the latest v115 TomatoUSB by Shibby if that matters.

    :mad::mad::mad::mad::mad::mad::mad::mad::mad::mad::mad::mad::mad::mad::mad:
     
  26. PeterT

    PeterT LI Guru Member

    What line ending is notepad++ set to?

    Sent from my Nexus 7 using Tapatalk 4
     
  27. shibby20

    shibby20 LI Guru Member

    @68rustang you have FAT32 partition
    this is propably your problem. Formatdrive to ext2 or ext3 and try again.

    BTW this script using ipset as i see. i checked Victek`s and Toastman`s sources and they don`t include ipset. Well it will not works on they firmwares. BUT it will works on RMerlin mod :) I think you should write this info in the first post.

    Best Regards
     
  28. 68rustang

    68rustang Network Newbie Member

    I pulled the USB drive and reformatted it with gparted to ext3. The drive is now named storage instead of USB.

    Notepad++ EOL is set to UNIX.

    Same results:
    Code:
    /tmp/.wxJfBUI7: line 5: /tmp/mnt/storage/p2partisan/p2partisan.sh: not found 

    IT WORKED! Well sort of...

    It has been sitting here for quite a while (30+ minutes)
    Code:
    loading modules
    loading ports 80,443,993,25,21 exemption
    loading the whitelist
    loading blacklist 1 - level1
    loading blacklist 2 -
    
    I have all lists except level1 commented out.
     
    Last edited: Nov 24, 2013
  29. rs232

    rs232 LI Guru Member

    Good stuff. About the 30 minutes: what router are you using (how much RAM)?

    P.S. if you have only level1 uncommented you shouldn't see the line called:
    Code:
    loading blacklist 2 -
    
     
  30. rs232

    rs232 LI Guru Member

    Thanks Shibby, I wasn't aware of this I'll update the first port as advised.

    Regards
     
  31. 68rustang

    68rustang Network Newbie Member

    I am running a Netgear WNR3500Lv2 with 128MB of RAM. With p2partisan and adblock running I have ~92MB free.

    That is why I mentioned it. My whitelist was as downloaded from pastebin with only level1 uncommented. I tried a minor edit this morning adding a space between the "#" and level2, level3 etc. It didn't think it would make a difference and it didn't. I also changed spywere to spyware.
    Code:
    # Generic lists
    level1 http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz
    # level2 http://list.iblocklist.com/?list=bt_level2&fileformat=p2p&archiveformat=gz
    # level3 http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=gz
    # edu http://list.iblocklist.com/?list=bt_edu&fileformat=p2p&archiveformat=gz
    # spyware http://list.iblocklist.com/?list=bt_spyware&fileformat=p2p&archiveformat=gz
    # spiders http://list.iblocklist.com/?list=bt_spider&fileformat=p2p&archiveformat=gz
    # hijacked http://list.iblocklist.com/?list=bt_hijacked&fileformat=p2p&archiveformat=gz
    # dshield http://list.iblocklist.com/?list=bt_dshield&fileformat=p2p&archiveformat=gz
    
    # Country lists available here
    # http://www.iblocklist.com/lists.php?category=country
    # Browse the URL above and pick up the list you want to block (if any)
    # e.g. Aruba
    # Aruba [URL]http://list.iblocklist.com/?list=aw&fileformat=p2p&archiveformat=gz[/URL]
     
  32. rs232

    rs232 LI Guru Member

    I've just replicated your scenario (single blacklist) and it works find from here.
    I still suspect your editor is playing a part here. Try to remove every empty line (including any at the end of the file)
    About the typo I'll correct that however it's only a name and does not affect the functionality. Thanks for spotting that
     
  33. 68rustang

    68rustang Network Newbie Member

    I don't think it is my editor because it was doing the same thing before I edited the file. What version of Tomato are you running?
     
  34. rs232

    rs232 LI Guru Member

    shibby 105 AIO on asus rt-n16
     
  35. bagu

    bagu LI Guru Member

    I have the same issue

    I use /tmp/mnt/sda1/p2partisan/p2partisan.sh
    /tmp/.wx0vLAOn: line 5: /tmp/mnt/sda1/p2partisan/p2partisan.sh: not found

    My SD card is in ext2

    My router : Asus RT-N66u
    My firmware : Tomatousb 1.28.0503.4 MIPSR2Toastman-RT-N K26 USB VLAN-Ext

    blacklist, p2partisan.sh and whitelist have been created succesfully.
    There is no error while the installation.

    EDIT : loading module don't work
    insmod: 'ip_set.ko': module not found
    insmod: 'ip_set_iptreemap.ko': module not found
    insmod: 'ipt_set.ko': module not found
     
    Last edited: Dec 5, 2013
  36. jerrm

    jerrm Addicted to LI Member

    Can't comment on the not found message, but looks like the script isn't there.

    Toastman doesn't have ipset. Only Shibby and Merlin.
     
  37. bagu

    bagu LI Guru Member

    Bad news...
     
  38. shadowro

    shadowro Network Newbie Member

    p2partisan.sh: not found

    My SD card is in ext2

    should I do ext3? or doesn't matter

    running on shibby 115 aio asus rt-n16

    thank you.
     
    Last edited: Dec 11, 2013
  39. shadowro

    shadowro Network Newbie Member

    /tmp/mnt/sdb3/
    is ext3

    still says not found.

    any help please? thank you.
     
    Last edited: Dec 27, 2013
  40. Josh Z.

    Josh Z. Network Newbie Member

    I noticed an issue in the initial installation code.
    Code:
    #The following line
    sed "s#/cifs1/p2partisan#$PWD#g" -i ./p2partisan.sh
    
    #should be
    sed "s#$INSTALLDIR/p2partisan#$PWD#g" -i ./p2partisan.sh


    If you receive the error "-sh: p2partisan.sh: not found" run the .sh script file like this "sh yourScript.sh"
    Code:
    #instead of runnig it like this
    /cifs1/p2partisan/p2partisan.sh
    
    #run it like this
    sh /cifs1/p2partisan/p2partisan.sh


    In case the script fails with "cant cd to..." on line 7, it is because of \r\n line endings.
    Close the editor you use to edit the file "p2partisan.sh" and run the command
    Code:
    tr -d "\r"< p2partisan.sh > p2partisan2.sh
    Now run "p2partisan2.sh" and it should work.
     
    Last edited: Dec 29, 2013
  41. Josh Z.

    Josh Z. Network Newbie Member

    Now I receive the following message "line 151: ./iptables-add-add: not found". Any ideeas on how to fix it?

    OFF TOPIC: 68rustang, can you please share with us your adblock script?
     
    Last edited: Dec 29, 2013
  42. shadowro

    shadowro Network Newbie Member

    used:
    sh /tmp/mnt/sdb3/p2partisan/p2partisan.sh

    got:
    /tmp/mnt/sdb3/p2partisan/p2partisan.sh: cd: line 7: can't cd to /tmp/mnt/sdb3/p2partisan

    /tmp/mnt/sdb3/p2partisan/p2partisan.sh: line 29:
    : not found
    /tmp/mnt/sdb3/p2partisan/p2partisan.sh: line 33:
    : not found
    /tmp/mnt/sdb3/p2partisan/p2partisan.sh: line 34:
    : not found
    loading modules

    /tmp/mnt/sdb3/p2partisan/p2partisan.sh: line 39: syntax error: unexpected word (expecting "do")

    any idea?
    thank you.
     
  43. Josh Z.

    Josh Z. Network Newbie Member

    shadowro, it is because of \r\n line endings. Check my previous reply.
     
  44. rs232

    rs232 LI Guru Member


    not sure what you are doing wrong, but please post the output of
    ls -la /tmp/mnt/sdb3/p2partisan
     
  45. Josh Z.

    Josh Z. Network Newbie Member

    One more hint: if you have a label set to the partition, try to use that instead of the name of the drive.
    For example I have a partition labeled "UsbStorage", so instead of /tmp/mnt/sda1/... I use /tmp/mnt/UsbStorage/.
     
  46. shadowro

    shadowro Network Newbie Member

    nevermind, got it working, but still got an wget error for level2 list

    thank you Josh Z.
     
    Last edited: Jan 7, 2014
  47. Josh Z.

    Josh Z. Network Newbie Member

    You should CD to the folder where you have the script p2partisan2.sh and run TR command there.

    And for the error "line 151: ./iptables-add-add: not found" is it normal for the -add to be 2 times? fixed, without mentioning the fix.

    rs232 Can you please tell me why on line 135 you put pos=`expr 13 + $counter` ?
    Why do you need to start after 13 indexes more?
    This is a big issue because I receive the error "iptables: Index of insertion too big."

    One more thing you might have to look into is --set being deprecated and replaced by --match-set.
    And you should really fix the issue with \r and \n in the file you are offering for wget.
     
    Last edited: Jan 10, 2014
  48. farfromovin

    farfromovin Network Newbie Member

    Thanks to Josh Z's suggestions, I've got past a few of the inital errors running the script. Now it loads the blacklists, to which I've added some of my own from iblocklist, but runs into problems with the iptables. Here is the terminal window:
    Code:
    loading blacklist 11 -
    Setting FORWARD iptables
    iptables: Index of insertion too big
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 5: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 6: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 7: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 8: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 9: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 10: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 11: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 12: src,dst: not found
    iptables v1.3.8: --set requires two args.
    Try `iptables -h' or 'iptables --help' for more information.
    ./iptables-add: line 13: src,dst: not found
    iptables: Index of insertion too big
    root@unknown:/tmp/mnt/sda/p2partisan#
    
    I'm very green to all this, but I love the idea and really want to get this sorted. If anyone can help out I'd appreciate it.
     
    Last edited: Jan 11, 2014
  49. Josh Z.

    Josh Z. Network Newbie Member

    iptables: Index of insertion too big it is because of the line 135: pos=`expr 13 + $counter`
    I removed the 3 and it seems to work, but I don't know exactly how to test it to be sure.

    "--set requires two args" is because some of your lists might contain some errors. I noticed on some of Squidblacklist they have some lines like this one ":BL-ADS,;; out;" which will break the script.
     
  50. farfromovin

    farfromovin Network Newbie Member

    Ok, I removed the "3" from line 135 and was able to run the script, although without my subscribed iblocklist url's. I'm going to play around with this some more an see if I can get my lists to run instead of the included, and need to find a way to test it as well. Thanks for the help!
    Code:
    root@unknown:/tmp/mnt/sda/p2partisan# sh p2partisan2.sh
    loading modules
    loading ports 80,443,993,25,21 exemption
    loading the whitelist
    loading blacklist 1 - level1
    Setting FORWARD iptables
    loading blacklist 2 -
    Setting FORWARD iptables
    root@unknown:/tmp/mnt/sda/p2partisan#
    So the next newb question is how do we get this to update it's list on say a weekly basis and does it run at boot or do we need to manually start the script each time?

    /edit
    Does the following log from the router look right?
    Code:
    Jan 12 03:13:27 unknown user.warn kernel: ip_set version 4 loaded
    Jan 12 03:14:23 unknown user.alert kernel: Blacklist-Dropped:IN=vlan2 OUT=br0 SRC=85.17.73.214 DST=192.168.1.XXX LEN=163 TOS=0x00 PREC=0x00 TTL=55 ID=32054 DF PROTO=TCP SPT=47529 DPT=8080 WINDOW=72 RES=0x00 ACK PSH URGP=0 
     
    Last edited: Jan 12, 2014
  51. Josh Z.

    Josh Z. Network Newbie Member

    The log is fine, you are good to go!
     
  52. farfromovin

    farfromovin Network Newbie Member

    I just have a lot of tweaking on the blacklist to do with different URLs. My script is running good now, thanks for all the assistance.
     
    Last edited: Jan 13, 2014
  53. Josh Z.

    Josh Z. Network Newbie Member

    What is very frustrating is that i did manage to make the script work, but I don't think it has the desired results. I am still new to iptables and I am trying to wrap my mind around it. The list I loaded contains IPs from china, but I still see in my log that IP's from china are trying to bruteforce my router on some ports. I checked the logged IP's and they are in the range of blocked IPs, so they should have been blocked.
    Code:
    Jan 13 16:22:13 dropbear[29088]: Login attempt for nonexistent user from 61.160.215.237:2117
    
    I did ask if there is a way to test the script, but it seems that rs232 went on vacation.

    What I am thinking now is to check my iptables after I run the script and compare with what it was before and try to understand the differences. If anybody knows more about iptables and can enlighten us, is free to leave a comment or even PM!

    The lines marked with [!] have been added after I run the script and BlockList4Router is the name of my merged list from iblocklist.com (yes, I am supporting them! :) )

    Code:
    # iptables --list
    Chain INPUT (policy ACCEPT)
    target     prot opt source               destination
    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:8443
    ACCEPT     udp  --  anywhere             anywhere             udp dpt:50003
    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:50003
    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:9091
    DROP       all  --  anywhere             anywhere             state INVALID
    ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
    ACCEPT     all  --  anywhere             anywhere             state NEW
    ACCEPT     all  --  anywhere             anywhere             state NEW
    ACCEPT     udp  --  anywhere             anywhere             udp spt:bootps dpt:bootpc
    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:8082
    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:https
    SSHBFP     tcp  --  anywhere             anywhere             tcp dpt:ssh state NEW
    DROP       all  --  anywhere             anywhere
    
    Chain FORWARD (policy DROP)
    target     prot opt source               destination
    TCPMSS     tcp  --  anywhere             anywhere             tcpflags: FIN,SYN,RST,ACK/SYN TCPMSS clamp to PMTU
    [!]LOGGING    all  --  anywhere             anywhere             match-set BlockList4Router src,dst
    [!]ACCEPT     tcp  --  anywhere             anywhere             multiport dports www,https
    [!]ACCEPT     tcp  --  anywhere             anywhere             multiport sports www,https
    ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
    [!]ACCEPT     all  --  anywhere             anywhere             match-set whitelist src,dst
    DROP       all  --  anywhere             anywhere
    DROP       all  --  anywhere             anywhere
    DROP       all  --  anywhere             anywhere             state INVALID
    ACCEPT     all  --  anywhere             anywhere
    DROP       icmp --  anywhere             anywhere
    ACCEPT     all  --  anywhere             anywhere             ctstate DNAT
    ACCEPT     all  --  anywhere             anywhere
    
    Chain OUTPUT (policy ACCEPT)
    target     prot opt source               destination
    
    Chain FUPNP (0 references)
    target     prot opt source               destination
    
    [!]Chain LOGGING (1 references)
    [!]target     prot opt source               destination
    [!]LOG        all  --  anywhere             anywhere             limit: avg 2/min burst 5 LOG level alert prefix "Blacklist-Dropped:"
    [!]DROP       all  --  anywhere             anywhere
    
    Chain PControls (0 references)
    target     prot opt source               destination
    ACCEPT     all  --  anywhere             anywhere
    
    Chain SSHBFP (1 references)
    target     prot opt source               destination
               all  --  anywhere             anywhere             recent: SET name: SSH side: source
    DROP       all  --  anywhere             anywhere             recent: UPDATE seconds: 60 hit_count: 4 name: SSH side: source
    ACCEPT     all  --  anywhere             anywhere
    
    Chain logaccept (0 references)
    target     prot opt source               destination
    LOG        all  --  anywhere             anywhere             state NEW LOG level warning tcp-sequence tcp-options ip-options prefix "ACCEPT "
    ACCEPT     all  --  anywhere             anywhere
    
    Chain logdrop (0 references)
    target     prot opt source               destination
    LOG        all  --  anywhere             anywhere             state NEW LOG level warning tcp-sequence tcp-options ip-options prefix "DROP"
    DROP       all  --  anywhere             anywhere
    
     
  54. jerrm

    jerrm Addicted to LI Member

    You can't stop them from trying, all the rules can do is drop the attempt before they get to an open port.

    That said, most port scans will show up against the router IP on the INPUT chain, unless the port is forwarded. This script does not do anything for that. It only adds rules to the FORWARD chain.
     
  55. Josh Z.

    Josh Z. Network Newbie Member

    Ok, I managed to make it working. Because I am using a custom list, the name of the list in blacklists file should be the value contained by the GET parameter "list" (which is equal to the name of the file inside the .gz archive).

    The problem I have now is that I don't know how to force the script to update the file. I download it once, but I want to be able to download an updated list after a while, as the list might be changed.

    Anyone... an idea?
     
  56. farfromovin

    farfromovin Network Newbie Member

    Josh, what if you used the scheduler function under the admin tab and had it run p2partisan.sh, say, once a week? By running the script, it reaches out to the URL's to build a new .gz archive correct?
     
  57. Josh Z.

    Josh Z. Network Newbie Member

    Not really. If you run the script for the first time, the list is downloaded. After that you can run the script as many times you want but the list (in .gz format) will not be redownloaded (because of shitty logic in the script). It doesn not even use timestamp option of wget.

    I will try to understand better the script and maybe come with a better version of it. Bash scripting is new for me and I am learning it now mainly because of this script.

    Edit: timestamp is not available on wget built for Asus RT-AC68U. I was able to make the script redownload the file everytime it is run, but now I am checking on how to refresh the new data into iptables.
     
    Last edited: Jan 13, 2014
    farfromovin likes this.
  58. jerrm

    jerrm Addicted to LI Member

    Don't criticize what you don't understand. There is no timestamp option in busybox's wget.

    With stock tomato, you have to write the code to read the last-modified header yourself before calling wget. Since many of the iblocklist url's end up with a redirect, be prepared to follow the redirects until you get a valid header, and then work out how to parse and compare the date, with busybox's crippled date command or some other method.

    It can all be done, but apparently not worth the effort for rs232 when a simple rm will force an update. Post an update, suggestion, fix or rewrite if you want, otherwise accept it and move on without making negative, unconstuctive comments.
     
    koitsu likes this.
  59. koitsu

    koitsu Network Guru Member

    Sadly there really isn't a good workaround for this problem with Busybox wget either. I was going to recommend doing a HEAD request and look at Content-Length + compare to filesize, but there's no way to do that aside from using nc manually + a while read statement (meaning: it's doable but it's ugly).

    Possibly Josh Z. should submit a request/ticket with the Busybox folks to implement such support. Understand however that this is somewhat tricky at times, because IMS (If-Modified-Since) and some other fields are not as simple as people make them out to be (the HTTP specification is complex in this regard). This comes into play greatly if caching proxies are involved.
     
  60. rs232

    rs232 LI Guru Member

    Wow this thread got more popular than I thought. Thanks for all the comment positive/negative, to me they all help towards writing a better script!
    I want to apologize not to have been that active here lately. The thing is that I'm an incurable backpacker and often I set up for a random destination for rather long time.
    I'm 4000 miles away from my laptop and to be fair don't really have to time to look into this until I'm back home (mid March I guess though I still don't have a return ticket as yet).
    That said, please be patient and do follow jerm's advice. If you find a better way to do things this is the right thread do discuss/share!

    I'll probably release an updated version as soon as I get back home following all the input received.

    Keep it up!

    peace and love :cool:
    rs232
     
  61. 68rustang

    68rustang Network Newbie Member

  62. darksky

    darksky Reformed Router Member

    @rs232 - Please host your code on github which allows for downloads and for others to contribute to your code. Accounts are free.
     
  63. rs232

    rs232 LI Guru Member

    Thanks for the advice, I will definitely look into it an post an update here asap
     
  64. rs232

    rs232 LI Guru Member

    Remember the p2partisan is applied on the forward chain! This is meant to protect LAN clients and not the router itself.
     
  65. rs232

    rs232 LI Guru Member

    v1.05 released, please try it out and let me know if any problem.

    This version should have resolved the iptables: Index of insertion too big occasional error message
    it also has a cleaner code.

    Thanks!
     
  66. rs232

    rs232 LI Guru Member

    v1.06 released. Please check the changelog, there are lots of crucial changes, especially the usage of the INPUT table to protect new packets as soon as the reach the router.

    Check out also the new troubleshooting instructions as per original post.

    Let me know if any problem
     
    Last edited: Mar 23, 2014
    Goggy and shadowro like this.
  67. farfromovin

    farfromovin Network Newbie Member

    Running the new version now rs232. I did have to do this though:
    tr -d "\r"< p2partisan.sh > p2partisan2.sh
    As my install couldn't cd to the correct folder during install. Other than that it seems alright. Thanks for the hard work and keep polishing it. This script really helps my setup!
     
    rs232 likes this.
  68. rs232

    rs232 LI Guru Member

    Interesting... It has to be related with the Windows EOL notepad++ uses by default.
    Try version 107 I've just published which btw allows you to enable/disable logs
     
    Last edited: Apr 14, 2014
  69. farfromovin

    farfromovin Network Newbie Member

    Thanks, I'll install it later today. Got a nasty gram from my ISP this morning so I need to take a look at what's going on anyway.

    Is there a better windows program I can use to edit the black/white lists?
     
  70. rs232

    rs232 LI Guru Member

    I'd say as long as you can specify the type UNIX EOL you should be fine... alternatively use nano from the shell (see optware) or vi if that does the job to you
     
  71. farfromovin

    farfromovin Network Newbie Member

    How can I enable the logs?
    from the OP: 3) If you have the option syslog=1 set (available from v1.07 and above)

    where can I set the option is what I'm getting at. Sorry for the newb learning curve here.

    -edit- I just input the commands as outlined and saw the connections dropped in SSH as well as on the routers logfile. Thanks!
     
  72. farfromovin

    farfromovin Network Newbie Member

    It appears that p2partisan stops running on my router for some reason. I set it up last night, verified it was dropping packets correctly. This afternoon, I checked the logs again and I get this:

    Code:
    root@DarkKnight:/tmp/home/root# iptables -L INPUT -v | grep P2PARTISAN
    root@DarkKnight:/tmp/home/root# cd /tmp/mnt/sda/p2partisan
    root@DarkKnight:/tmp/mnt/sda/p2partisan# iptables -L INPUT -v | grep P2PARTISAN
    root@DarkKnight:/tmp/mnt/sda/p2partisan# iptables -L P2PARTISAN-DROP -v
    iptables: No chain/target/match by that name
    I had to run the command sh p2partisan2.sh to get it to work again. Of note, I have not restarted the router and I do have the wanup script from the OP in place. I'm running Shibby's 116 on an N66U.

    -edit- looks like if I modify my wan up code slightly it starts on reboot
    Code:
    sleep 180
    sh /tmp/mnt/sda/p2partisan/p2partisan2.sh
    I'll continue to monitor it over the next few days to verify the iptables stay loaded. I don't know why it wouldn't. Sorry for the newbness again guys.
     
    Last edited: Apr 18, 2014
  73. rs232

    rs232 LI Guru Member

    This is a common question on tomato and scripts stored on external media. In Linux in general if you need to specify the shell before the script it probably means that the storage where the script sits has been mounted with a --noexec parameter. Verify running "mount"

    About the other issue. not sure why it would stop, but I would indeed double check the uptime even though you didn't reboot (you never know).

    Finally (I'm sure I'm repeating myself but you never know it might help you) make sure the location where the script lives is fully mounted and available to tomato before the command you mentioned above is run.

    HTH
    rs232

    P.S. Once you're happy with the script and how it runs do consider set syslog option to 0.
     
  74. farfromovin

    farfromovin Network Newbie Member

    rs232- Thanks for the advice, every bit helps. So, my uptime is 14hrs and a few minutes after last boot I looked at the logs and it was running. Now, it is not so that means it has stopped sometime in the last 12 hours. The media is mounted still, I've never had a problem with it not mounted. Should this be installed internally? My p2partisan folder is only 17kb.
     
  75. jerrm

    jerrm Addicted to LI Member

    Something probably caused the firewall to reset and eliminated the rules. This can happen without wanup being executed. Probably need a hook in the firewall script section of the GUI or with a .fire scripts.
     
  76. farfromovin

    farfromovin Network Newbie Member

    Hmm, well I just rebooted and let the wanup script execute so we'll see if it hangs in there with that route instead of by manual execution. If not, I'll look into the firewall script. Thanks for the tip.
     
  77. jerrm

    jerrm Addicted to LI Member

    You do not want to run the entire script from the firewall gui. The ipsets will survive a "service firewall restart" or the internal equivalent. Only the actual rules need to be re-established.
     
  78. rs232

    rs232 LI Guru Member

    Also, if you have a powerful device do consider increasing the log size to something big e.g. 5M having logging enabled on P2Partisan at the same time. You should be able to find something in the logs not necessarily related to P2Partisan that might help you understand what's going on
     
  79. farfromovin

    farfromovin Network Newbie Member

    Thanks, I have. But, its been blocking for 6 days now so I'm sure it must have been something I did wrong before.
     
  80. farfromovin

    farfromovin Network Newbie Member

    I must have spoke too soon. Somewhere between days 6 and 8 of uptime p2partisan stopped? At this point I'm not sure what is going wrong here. I don't have anything running on this router besides p2partisan, and I'm on Tomato 1.28 by Shibby build 116 on a RT-N66U. The script is installed on a micro sd card in the router formatted ext2.
     
  81. rs232

    rs232 LI Guru Member

    Humm strange... when you say stopped what do you actually mean?
    Does the P2Ppartisan table exist?
    What is the uptime of the device?
    What is the connection uptime?
    Is there really anything in the logs that might help?
     
  82. rs232

    rs232 LI Guru Member

    I've just noticed that starting/stopping the QoS service does indeed remove P2Partisan!
    This shouldn't happen, but have you enabled/disabled QoS?
     
  83. jerrm

    jerrm Addicted to LI Member

    Yes it should. It will cause the equivalent of a "service firewall restart." As I stated earlier, for the rules to survive there needs to be a hook into the firewall either in the gui or with an autorun file to re-establish the P2P rules. There are many things that cause a firewall restart that do not run wan-up code.

    Note the ipsets themselves probably survived the restart, only the rules got wiped.
     
  84. farfromovin

    farfromovin Network Newbie Member

    I haven't started or stopped QOS, but I did start and stop the Captive Portal.

    When I follow the promtps in the first post, nothing pulls up. When I look at the logs in the router GUI, there are no blacklist drops when there should be. The connection uptime is always the same as the uptime of the device.
    Code:
    root@DarkKnight:/tmp/home/root# cd /tmp/mnt/sda/p2partisan
    root@DarkKnight:/tmp/mnt/sda/p2partisan# iptables -L INPUT -v | grep P2PARTISAN
    root@DarkKnight:/tmp/mnt/sda/p2partisan# iptables -L P2PARTISAN -v
    iptables: No chain/target/match by that name
    root@DarkKnight:/tmp/mnt/sda/p2partisan# iptables -L P2PARTISAN-DROP -v
    iptables: No chain/target/match by that name
    root@DarkKnight:/tmp/mnt/sda/p2partisan#
    
     
    Last edited: Apr 28, 2014
  85. jerrm

    jerrm Addicted to LI Member

    Again, I would consider that 100% normal and expected behavior - the portal stop/start caused a firewall restart. Such should be expected with most config changes.
     
    rs232 likes this.
  86. rs232

    rs232 LI Guru Member

    Good point jerrm! You also gave me the solution :)
    Let's run P2Partisan from the script/firewall!
    I tested it, it works!

    Thanks for spotting this out farfromovin!

    I've updated the installation procedure to add P2Partisan in the Administration/Scripts/Firewall field
     
    Last edited: Apr 28, 2014
  87. jerrm

    jerrm Addicted to LI Member

    The firewall code can be called multiple times during startup, potentially a lot of wasted effort downloading and re-building lists multiple times. Best to only update the rules in the firewall section and run the rest from init.
     
  88. rs232

    rs232 LI Guru Member

    I understand but being this an iptables operation only (as ipset is untouched) it's not a big deal in my opinion. In fact if you run the scrip the second time you'll notice that it takes almost no resources/time to run.

    I understand this is a lazy answer, but otherwise the alternative would be to have a look into any exception that make the P2PARTISAN iptable disappear.

    Perhaps in the next version I could implement an easy control rule to skip the execution if the P2PARTISAN already iptable exists.
    I also want to implement an initd syntax compatible way to run the script like
    p2partisan.sh start|stop|restart|force|update

    Thanks for the input! It's nice to have somebody double-checking things :)

    rs232
     
  89. rs232

    rs232 LI Guru Member

    P2Partisan 1.08 released
    - Substantial script redesign with usage of routines
    - Changes in the installation process
    - Added support for parameters start|stop|restart|status|update
    - Added automatic update management via crontable
    - Added control to skip start if P2Partisan is already running
    - Improved usage of the log

    please read well the first post as this time there are significant improvement/changes.

    let me know if any problem
    rs232
     
  90. shadowro

    shadowro Network Newbie Member

  91. rs232

    rs232 LI Guru Member

    Thanks for the feedback. I don't get it... I did open the script from the shell and all the EL are fine on my system.
    So there should be no need to go through the steps you described. Unless I'm missing something, hummm...
    I know it's up and running now but if you run the installation as per post 1 changing simply the installation path it should work (?)

    Thanks!
    rs232
     
  92. shadowro

    shadowro Network Newbie Member

    like Josh Z. says:
    #The following line
    sed "s#/cifs1/p2partisan#$PWD#g" -i ./p2partisan.sh

    #should be
    sed "s#$INSTALLDIR/p2partisan#$PWD#g" -i ./p2partisan.sh

    had to do:
    tr -d "\r"< p2partisan.sh > p2partisan2.sh
    I remember it gave me something like: file not found or error opening file
    and the tr command fixed it

    had to also modify the script from here:
    http://pastebin.com/raw.php?i=uqKsQM0r

    so the line:
    P2Parisandir=/cifs1/p2partisan
    will be, in my case:
    P2Parisandir=/tmp/mnt/OPTWARE/p2partisan

    hope it helps.
     
  93. rs232

    rs232 LI Guru Member

    Ok yes and no.
    no:
    You don't have to modify anything other than one line, and specifically the INSTALLDIR variable at the beginning of in the installation script. Everything else is adjusted automatically. That's the way it was design and the way it works (on my system at least). That sed line gets the working directory where the command is run and replaces the default /cifs1. So it's correct the way it is.

    yes:
    I will have a look at the tr command, I thought to have it solved but it doesn't look like apparently. leave it with me on this :)

    thanks!
    rs232
     
  94. rs232

    rs232 LI Guru Member

    Ok I've modified the installation part adding the tr command at the installation point, so that should solve the problem.
    I've also added a section to upgrade an existing installation.

    Thanks for reporting the problem!

    rs232
     
  95. rs232

    rs232 LI Guru Member

    version 2.00 is out, as usual refer to the initial post.
    Any feedback is welcome :)

    rs232
     
    Goggy and may like this.
  96. farfromovin

    farfromovin Network Newbie Member

    Cool, I'll update and test tomorrow!
     
  97. rs232

    rs232 LI Guru Member

    Version 2.10 released with some extra improvements and minor bugfix
     
    Goggy likes this.
  98. Kim K

    Kim K Reformed Router Member

    Shouldn't the instructions say: (note the .sh on the end)?

    chmod -R 777 ./p2partisan.sh

    EDIT: for the upgrade
     
  99. rs232

    rs232 LI Guru Member

    Yes you're right I'll add it. I also updated the update procedure... just a typo ;-)
     
  100. The Master

    The Master Addicted to LI Member

    Hello,

    is it Possible to add this Lists in the Blocklist File?!

    Or is there a Problem with the Format?!

    Thanks for the Great Script
     

Share This Page