1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

P2Partisan [v5.14/v6.09] mass IP blocking - peerblock/peerguardian for tomato

Discussion in 'Tomato Firmware' started by rs232, Oct 11, 2013.

  1. rs232

    rs232 Network Guru Member

    I think Windows telemetry works on port 80 so no much you can done a port basis nor you want! But you canblock the IPs/FQDNs. All you need to do is to add them into the blacklist-custom file and restart p2partisan.
    As I can see from your screenshot you have 71 black references already and 14628 (?) white references + the public level1 black list is that right?
     
  2. glennsamuel32

    glennsamuel32 Network Newbie Member

    That is the confusing part...
    My custom-Blacklist with 17,000 ranges + the level1 = Black-ip of 323 ??
    And my whitelist has just 3 domains but White-ip shows 30,290 ??

    [​IMG]
     
  3. rs232

    rs232 Network Guru Member

    The references are what ipset tells

    e.g.:

    Code:
    ipset -L blacklist-custom 2> /dev/null | grep -E "(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])" | wc -l
    public lists are not included in the calculation. Also a reference can be a subnet and not necessarely an individual IP number.
     
    Last edited: Jun 21, 2017
  4. rs232

    rs232 Network Guru Member

    Inbound and Outbound are used for packets sourced and addressed to the router only. Forward catches (v)LAN activity.
     
  5. glennsamuel32

    glennsamuel32 Network Newbie Member

    Thanks for the explanation...

    Also in the level1.cidr, the first line starts with :
    create level1 hash:net family inet hashsize 131072 maxelem 4096000

    Is there anyway to customize the hash size ?
    What are the maximum sizes allowed ?
    I use the R8000 with 128 mb nvram...
     
  6. glennsamuel32

    glennsamuel32 Network Newbie Member

    On the ipset man page...

    hashsize
    This parameter is valid for the create command of all hash type sets. It defines the initial hash size for the set, default is 1024. The hash size must be a power of two, the kernel automatically rounds up non power of two hash sizes to the first correct value. Example:
    ipset create test hash:ip hashsize 1536
     
  7. rs232

    rs232 Network Guru Member

    That's right so the hash size is automatic, nothing needs to be tweaked.
     
  8. rs232

    rs232 Network Guru Member

    1 hashsize != 1 element

    P2Partisan does work out of the box no need to modify anything at the ipset level.
     
  9. glennsamuel32

    glennsamuel32 Network Newbie Member

    Thanks for your help !!
     
  10. Bird333

    Bird333 Network Guru Member

    Bump
     
  11. rs232

    rs232 Network Guru Member

    As mentioned multipe times this is bottom of my list. Nothing personal trust me but I truly believe the download from iblocklist to be very reliable. I'm way more concerned about filtering VPN traffic which at the moment works only outbound and can't understand why.
    If you're keen into this requets of your, why don't you give it a go yourself modifying the scriopt and perhaps posting here your achievement/issues you run into so that we can get there together?

    Thanks
     
  12. NutsN'bolts

    NutsN'bolts New Member Member

    Wow, ill give this a try. Sounds very usefull !!
     
  13. glennsamuel32

    glennsamuel32 Network Newbie Member

    Everything works well now...
    But there are constant connections on port 80 and 443...

    Can you think of a way to block these connections ?

    [​IMG]
     
  14. rs232

    rs232 Network Guru Member


    You do not want to filter at port level when it comes to 80 and 443.
    So add the IPs or even better FQDNs in the blacklist-custom file and run ./p2partisansh restart
    NOTE: is these are the windows10 default connection they are very likely to change throughought time.
     
  15. Bird333

    Bird333 Network Guru Member

    I wouldn't have a clue how to do this. Just looking at this script gives me headaches. :) I understand it's a low priority, but it's been over a year since it was brought up. Do you think you can look at it after you solve the VPN issue? Thanks!
     
  16. glennsamuel32

    glennsamuel32 Network Newbie Member

    Is it correct that only lists compressed in gz are valid under "blacklists" ?
    If so, will you consider lists in plain / raw text also ?
     
  17. rs232

    rs232 Network Guru Member

    Blacklists are the public blacklists and they are pretty much always provided in .gz format.
    What are these others clear tex lists you're referring to? Do you have an example?
     
  18. glennsamuel32

    glennsamuel32 Network Newbie Member

  19. rs232

    rs232 Network Guru Member


    I'm not aware of the iblocklists being outdated, regardless: no, P2Partisan does assume a specific format to be fed to it.
    This format being a text file compressed in .gz and the text organised in the format:

    Description:FromIP-ToIP

    e.g.
    China Internet Information Center (CNNIC):1.2.4.0-1.2.4.255

    I'll see what I can do to add support for the the raw and netset formats out of the box as it seems like a good idea.
     
  20. rs232

    rs232 Network Guru Member

    Pulled out due to a bug



    P2Partisan
    v6.10 is out

    NOTE: this is a consistent modification from the previous version.
    A reinstallation is suggested, but if you don't want to do that, make sure you run the following commands from within the current p2partisan folder:
    Code:
    1)
    rm /tmp/deaggregate.sh
    ./p2partisan.sh upgrade
    
    2) [optional] wget -O blacklists "https://pastebin.com/raw/ARx7NAYz"
    or add the relevant new lists e.g. raw/netset format manually before starting p2parisan again
    
    3) ./p2partisan.sh

    Changelog:
    - corrected list update minor issue
    - support for raw and netset style lists (thanks @glennsamuel32 )
    - introduced control to avoid updating lists the the list URL becomes unavailable or the URL simply doesn't exists (thanks @Bird333 )
    - optimisations where possible
    - added extra tutor control against empty primary lists (tutor will try to populate the ipset)
    - modification to allow similar named lists to cohexists (e.g. "level1" and "superduper_level1")
    - adjustments to the deaggregate.sh procedure

    As usual let us know if any issue/bug
     
    Last edited: Jul 15, 2017
  21. sszpila

    sszpila Reformed Router Member

    Is this normal? p2partisan cannot populate blocklists?
    Code:
    login as: root
    root@10.2.1.1's password:
    
    
    Tomato v1.28.0000 -132 K26ARM USB AIO-64K
     ========================================================
     Welcome to the Asus RT-AC56U [Pomidor]
     Uptime:  12:45:47 up 41 days, 21:11
     Load average: 0.11, 0.13, 0.13
     Mem usage: 13.6% (used 33.88 of 249.64 MB)
     WAN : ##.##.##.##/22 @ ##:##:##:##:##:##
     LAN : 10.2.1.1/24 @ DHCP: 10.2.1.5 - 10.2.1.20
     WL0 : I czego tu?! @ channel: SG8 @ ##:##:##:##:##:##
     WL1 : I czego tu?! @ channel: SG153 @ ##:##:##:##:##:##
     ========================================================
    
    root@Pomidor:/tmp/home/root# cd /cifs1/p2partisan/
    root@Pomidor:/cifs1/p2partisan# ls
    blacklist-custom     list.advertisement   list.edu             list.level1          list.spywere         whitelist
    blacklists           list.dshields        list.firehol_level1  list.level2          p2partisan.sh
    root@Pomidor:/cifs1/p2partisan# ./p2partisan.sh
    
    +------------------------- P2Partisan --------------------------+
    |                 _______ __               __
    |                |     __|  |_.---.-.----.|  |_
    |                |__     |   _|  _  |   _||   _|
    |                |_______|____|___._|__|  |____|
    |
    +---------------------------------------------------------------+
    +--------- PREPARATION --------
    | Loading the ipset modules
    +---- CUSTOM IP BLACKLIST -----
    | preparing blacklist-custom ...
    | Loading Blacklist_00 data ---> ***Custom IP blacklist***
    +--------- GREYPORTs ----------
    |  TransmissionBT:  Off
    +--------- WHITEPORTs ---------
    | Loading white TCP ports 80,443,3658,8080,5939
    | Loading white UDP ports 1194:1197,53,123,1723,3658,5939,67,68
    +--------- GREY IPs ---------
    | preparing IP greylist ...
    | Loading IP greylist data ---> ***IP greylist***
    +--------- WHITE IPs ---------
    | preparing IP whitelist ...
    | Loading IP whitelist data ---> ***IP Whitelist***
    +------- IP BLACKLISTs -------
    ./p2partisan.sh: line 1892: file: not found
    | Async loading [computed/slow] Blacklist_01 --> ***level1***
    ./p2partisan.sh: line 1892: file: not found
    | Async loading [computed/slow] Blacklist_02 --> ***level2***
    ./p2partisan.sh: line 1892: file: not found
    | Async loading [computed/slow] Blacklist_03 --> ***edu***
    ./p2partisan.sh: line 1892: file: not found
    | Async loading [computed/slow] Blacklist_04 --> ***spywere***
    ./p2partisan.sh: line 1892: file: not found
    | Async loading [computed/slow] Blacklist_05 --> ***advertisement***
    ./p2partisan.sh: line 1892: file: not found
    | Async loading [computed/slow] Blacklist_06 --> ***dshields***
    ./p2partisan.sh: line 1892: file: not found
    | Async loading [computed/slow] Blacklist_07 --> ***firehol_level1***
    | P2PARTISAN: ... P2Partisan started
    +------------------------- Controls ----------------------------+
    | P2PARTISAN: log-async found under dnsmasq -> OK
    +---------------------------------------------------------------+
    
    +------------------------- P2Partisan --------------------------+
    |                _______         __
    |               |_     _|.--.--.|  |_.-----.----.
    |                 |   |  |  |  ||   _|  _  |   _|
    |                 |___|  |_____||____|_____|__|
    |
    +-------------------------- Scheduler --------------------------+
    | P2PARTISAN: P2Partisan tutor is ON
    +---------------------------------------------------------------+
    root@Pomidor:/cifs1/p2partisan# ./p2partisan.sh tutor
    | P2PARTISAN: P2Partisan found the list level1 empty. Forcing ipset population
    
    +------------------------- P2Partisan --------------------------+
    |  _____   __         __                         __         __
    | |     |_|__|.-----.|  |_ ______.--.--.-----.--|  |.---.-.|  |_.-----.
    | |       |  ||__ --||   _|______|  |  |  _  |  _  ||  _  ||   _|  -__|
    | |_______|__||_____||____|      |_____|   __|_____||___._||____|_____|
    |                                     |__|
    |
    +---------------------------------------------------------------+
    |            background updating list: level1
    +---------------------------------------------------------------+
    ./p2partisan.sh: line 1890: file: not found
    
    +------------------------- P2Partisan --------------------------+
    |                _______         __
    |               |_     _|.--.--.|  |_.-----.----.
    |                 |   |  |  |  ||   _|  _  |   _|
    |                 |___|  |_____||____|_____|__|
    |
    +---------------------------------------------------------------+
    | P2Partisan up and running. The tutor is happy
    +---------------------------------------------------------------+
    root@Pomidor:/cifs1/p2partisan# ./p2partisan.sh status
    
    +------------------------- P2Partisan --------------------------+
    |            _______ __          __
    |           |     __|  |_.---.-.|  |_.--.--.-----.
    |           |__     |   _|  _  ||   _|  |  |__ --|
    |           |_______|____|___._||____|_____|_____|
    |
    | Release version:  v6.10 (13/07/2017)
    +---------------------------------------------------------------+
    |         Running:  Yes
    |         Autorun:  Yes
    |           Tutor:  Yes / 0 problems in the last 24h
    |        Debugger:  Off
    | Partisan uptime:  0d - 00:00:45
    |    Startup time:   seconds
    |      Dropped in:  0
    |    Rejected out:  0
    +---------------------------------------------------------------+
    |       Black IPs:  0
    |        Grey IPs:  0
    |       White IPs:  2 / 2 LAN IP ref defined
    |  TransmissionBT:  Off
    | White ports TCP:  80,443,3658,8080,5939
    | White ports UDP:  1194:1197,53,123,1723,3658,5939,67,68
    |    Blacklist_01:  [e] [e] [e] [o] -    8 KB - level1
    |    Blacklist_02:  [e] [e] [e] [o] -    8 KB - level2
    |    Blacklist_03:  [e] [e] [e] [o] -    8 KB - edu
    |    Blacklist_04:  [e] [e] [e] [o] -    8 KB - spywere
    |    Blacklist_05:  [e] [e] [e] [o] -    8 KB - advertisement
    |    Blacklist_06:  [e] [e] [e] [o] -    8 KB - dshields
    |    Blacklist_07:  [e] [e] [e] [o] -    8 KB - firehol_level1
    |                    ^   ^   ^   ^
    |      maxload: 2 - pri sec cid ipt - [e]mpty [l]oading l[o]aded [p]artial [q]ueued
    |    Consumed RAM:  116 KB
    +----------------------- Logs max(1/hour) ----------------------+
    | Jul 13 12:27:51 I=vlan2 O=br0 S=202.142.86.237 D=10.2.1.49 UDP S=12773 D=46882
    | Jul 13 12:23:12 I=br0 O=vlan2 S=10.2.1.49 D=47.202.18.233 UDP S=46882 D=6881
    +---------------------------------------------------------------+
    root@Pomidor:/cifs1/p2partisan#
    
    ---edit

    Nevermind, that was an old version of deaggregate.sh file. I deleted /tmp/deaggregate.sh file and run p2partisan.sh update.
    But still, blacklists are loaded, but size each of it is 8 KB and every cidr file contains "add {list name} 1.1.1.1"
     
    Last edited: Jul 13, 2017
  22. rs232

    rs232 Network Guru Member

    That shouldn't happen. Is this an upgrade or a fresh installation you did?
     
  23. sszpila

    sszpila Reformed Router Member

    First upgrade, second fresh install. And still p2partisan reports blocklists size 8 KB after p2partisan update and p2partisan tutor.
     
  24. rs232

    rs232 Network Guru Member

    hum, can you try to remove "/tmp/deaggregate.sh"
    If you still experience the issue it might be a bug.

    BTW in your post above it complaines about a file: not found I'm wondering where's that coming from as it works fine here.
     
  25. sszpila

    sszpila Reformed Router Member

    Yes, I removed one more time this file. The "file not found" disappeared, but blacklist's cidr files contains only one line : "add list name 1.1.1.1"

    Wysłane z mojego 2014811 przy użyciu Tapatalka
     
  26. rs232

    rs232 Network Guru Member

    Ok let me take the safe approach here.

    I'll revert the publich back to 6.08 and publish this version under the beta (so available under upgrade-beta) until this is resolved.

    Thanks for the feedback!
     
  27. Bird333

    Bird333 Network Guru Member

    I'm not sure based on your description. Is there some kind of error message displayed on the status page if a list didn't download or if a rule didn't get created for a list? Just curious, did you figure out your vpn issue?
     
  28. rs232

    rs232 Network Guru Member

    @Bird333

    I've rolled everything back to v6.08 and modified that version. The latest stable is now v6.09 which has the following modification only:

    - introduced control to skip list update when the URL becomes unavailable or it simply doesn't exists (thanks @Bird333 )

    I'll see when I can track the bug in the new list types later on. I bet it's nothing serious but need time to troubleshoot.
    No news on the VPN side and I will need some help to fully understand what's going on. Just waiting for some expert to get involved...
     
  29. sszpila

    sszpila Reformed Router Member

    Everything works fine with this version except one thing. I must comment firehol_level1 blacklist, because with this list enabled the level1 blocklist displays red "e" in ipt column on status display.

    Wysłane z mojego 2014811 przy użyciu Tapatalka
     
  30. rs232

    rs232 Network Guru Member


    Correct that's because the new blacklist still has references to raw and netset which are not supported on 6.09. As mentioned in the above post of mine I will reintroduce shortly, for the time being comment out any list that is not from the iblocklist provider
     

Share This Page