{"id":1642,"date":"2017-03-10T06:16:55","date_gmt":"2017-03-10T06:16:55","guid":{"rendered":"http:\/\/cdblog.cdstealer.com\/?p=1642"},"modified":"2024-12-19T17:18:00","modified_gmt":"2024-12-19T17:18:00","slug":"iptables-with-ipsets","status":"publish","type":"post","link":"https:\/\/cdblog.cdstealer.com\/?p=1642","title":{"rendered":"IPTables with IPSets"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

First thing is to ensure the correct options are set in the kernel.<\/p>\n\n\n\n

IP sets support the following type of sets:<\/p>\n\n\n\n

bitmap:ip<\/td>\nThe bitmap:ip set type uses a memory range, where each bit represents one IP address and can store up to 65535 (B-class network) entries. You can store same size network addresses in this kind of sets as well and an IP address will be in the set if the network address it belongs to can be found in the set.<\/td><\/tr>\n
bitmap:ip,mac<\/td>\nThe bitmap:ip,mac set type uses a memory range, where each 8 bytes represents one IP and a MAC addresses. A bitmap:ip,mac set type can store up to 65535 (B-class network) IP addresses with MAC.<\/td><\/tr>\n
bitmap:port<\/td>\nThe bitmap:port set type uses a memory range, where each bit represents one TCP\/UDP port. A bitmap:port type of set can store up to 65535 ports.<\/td><\/tr>\n
hash:ip<\/td>\nThe hash:ip set type uses a hash to store IP addresses where clashing is resolved by storing the clashing elements in an array and, as a last resort, by dynamically growing the hash. Same size network addresses can be stored in an hash:ip type of set as well.<\/td><\/tr>\n
hash:net<\/td>\nThe hash:net set type also uses a hash to store CIDR netblocks, which may be of different sizes. The same techique is used to avoid clashes as at the hash:ip set type.<\/td><\/tr>\n
hash:ip,port<\/td>\nThe hash:ip,port is similar to hash:ip but you can store IP address and protocol-port pairs in it. TCP, SCTP, UDP, UDPLITE, ICMP and ICMPv6 are supported with port numbers\/ICMP(v6) types and other protocol numbers without port information.<\/td><\/tr>\n
hash:ip,port,ip<\/td>\nYou can store IP address, port number, and IP address triples in an hash:ip,port,ip type of set.<\/td><\/tr>\n
hash:ip,port,net<\/td>\nYou can store IP address, port number and network address triples in this kind of set.<\/td><\/tr>\n
hash:net,port<\/td>\nThe set type supports to store network address and port number pairs.<\/td><\/tr>\n
hash:net,iface<\/td>\nIn this kind of set one can store network address and interface name pairs.<\/td><\/tr>\n
list:set<\/td>\nIn a list:set kind of set you can store other sets; it is like an ordered union of different sets.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
-*- Networking support --->\n    Networking options --->\n        [*] Network packet filtering framework (Netfilter) --->\n            <*> IP set support --->\n                (256) Maximum number of IP sets  \n                <*> bitmap:ip set support\n                <*> bitmap:ip,mac set support\n                <*> bitmap:port set support\n                <*> hash:ip set support\n                <*> hash:ip,mark set support\n                <*> hash:ip,port set support\n                <*> hash:ip,port,ip set support\n                <*> hash:ip,port,net set support\n                <*> hash:ip,mac set support\n                <*> hash:mac set support\n                <*> hash:net,port,net set support\n                <*> hash:net set support\n                <*> hash:net,net set support\n                <*> hash:net,port set support\n                <*> hash:net,iface set support\n                <*> list:set set support<\/code><\/pre>\n\n\n\n
If you're having to now add this, you'll obviously need to reboot once you have built and installed the updated kernel.<\/p>\n\n\n\n
Now let's install  ipset.<\/p>\n\n\n\n
# emerge -av ipset\n\nThese are the packages that would be merged, in order:\n\nCalculating dependencies... done!\n[ebuild R ] net-firewall\/ipset-6.29::gentoo USE=\"-modules\"<\/code><\/pre>\n\n\n\n
Ipset has the modules flag set by default.  With the above kernel config, the build will fail as I haven't used modules DUH!  So just disable the flag.  If you've build the kernel with modules instead, just leave as is.<\/p>\n\n\n\n
So on to the fun stuff.  Ipset has a pretty good help section and the manpage is detailed :)<\/p>\n\n\n\n
I use systemd, so there is no service to restart for iptables or ipset.<\/p>\n\n\n\n
Obviously you'll have iptables configured and running :)  By default I block everything and only have specific ports open to the world.  But that doesn't mean we don't need ipsets.  I'm going to use ipset to block spiders (crawlers) that I don't want or that bypass my robots.txt.  I could limit the blocking to specific ports eg 80 & 443, but I'm just going to block everything.  Just because I can ;)  I've also set a timeout (optional) so the IP's can be rotated and the sets don't grow too big.<\/p>\n\n\n\n
So let's create an ipset.  I only want to block CIDR, so I don't need to add port, mac etc etc.<\/p>\n\n\n\n
ipset create netSpiders hash:net timeout 86400<\/code><\/pre>\n\n\n\n
You can add IPs in here also, but I like to keep things clean so I also have:<\/p>\n\n\n\n
ipset create ipSpiders hash:ip timeout 86400<\/code><\/pre>\n\n\n\n
NOTE:  If you haven't rebooted into the ipset enabled kernel, you'll get the error:<\/p>\n\n\n\n
ipset v6.29: Kernel error received: set type not supported<\/code><\/pre>\n\n\n\n
You can list the set and any rules within by executing:<\/p>\n\n\n\n
# ipset list\nName: spiders\nType: hash:net\nRevision: 6\nHeader: family inet hashsize 1024 maxelem 65536 timeout 86400\nSize in memory: 368\nReferences: 0\nNumber of entries: 0\nMembers:<\/code><\/pre>\n\n\n\n
As you can see, we have nothing defined yet.  So let's add some spiders.  Unfortunately it's per CIDR.<\/p>\n\n\n\n
# ipset add netSpiders 123.151.148.0\/22\n# ipset add netSpiders 157.54.0.0\/15\n# ipset add netSpiders 157.56.0.0\/14\n# ipset add netSpiders 157.60.0.0\/16\n# ipset add netSpiders 65.52.0.0\/14\n# ipset add netSpiders 5.10.83.0\/25\n# ipset add netSpiders 208.115.113.80\/28\n# ipset add netSpiders 208.115.111.64\/28\n# ipset add netSpiders 198.27.64.0\/18\n# ipset add netSpiders 54.160.0.0\/12\n# ipset add netSpiders 54.224.0.0\/12<\/code><\/pre>\n\n\n\n
We can then list our set to see the rules.<\/p>\n\n\n\n
# ipset list netSpiders\nName: netSpiders\nType: hash:net\nRevision: 6\nHeader: family inet hashsize 1024 maxelem 65536 timeout 86400\nSize in memory: 1072\nReferences: 0\nNumber of entries: 11\nMembers:\n54.224.0.0\/12\n208.115.113.80\/28\n54.160.0.0\/12\n123.151.148.0\/22\n198.27.64.0\/18\n5.10.83.0\/25\n157.60.0.0\/16\n208.115.111.64\/28\n157.56.0.0\/14\n65.52.0.0\/14\n157.54.0.0\/15<\/code><\/pre>\n\n\n\n
You can remove rules by executing:<\/p>\n\n\n\n
# ipset del netSpiders 54.224.0.0\/12<\/code><\/pre>\n\n\n\n
We then add the ipset to iptables.  I log dropped connections, but if you are just dropping, then just use DROP as the target instead.<\/p>\n\n\n\n
iptables -A INPUT -m set --match-set netSpiders src -j LOGGING<\/code><\/pre>\n\n\n\n
Looking at my firewall log, I can see the ipset already working.<\/p>\n\n\n\n
kernel: Incoming Dropped: IN=enp2s0 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:08:00 SRC=54.227.98.207 DST=xxx.xxx.xxx.xxx LEN=60 TOS=0x00 PREC=0x00 TTL=43 ID=27451 DF PROTO=TCP SPT=56227 DPT=80 WINDOW=14600 RES=0x00 SYN URGP=0<\/code><\/pre>\n\n\n\n
If you check iptables, this is what you'll see for the ipset entry.<\/p>\n\n\n\n
# iptables -L -n\n...\nLOGGING all -- 0.0.0.0\/0 0.0.0.0\/0 match-set netSpiders src\n...<\/code><\/pre>\n\n\n\n
You can remove ipsets completely by executing:<\/p>\n\n\n\n
ipset destroy netSpiders<\/code><\/pre>\n\n\n\n
not specifying the set will delete ALL sets.<\/p>\n\n\n\n
If a set is in use by iptables, you will get the following error.<\/p>\n\n\n\n
ipset v6.29: Set cannot be destroyed: it is in use by a kernel component<\/code><\/pre>\n\n\n\n
Also ipsets, like iptables, are not persistent from reboots.  You can save the sets by executing:<\/p>\n\n\n\n
ipset save > ipset.save<\/code><\/pre>\n\n\n\n
# cat ipset.save \ncreate netSpiders hash:net family inet hashsize 1024 maxelem 65536 timeout 86400\nadd netSpiders 157.60.0.0\/16\nadd netSpiders 65.52.0.0\/14\nadd netSpiders 208.115.113.80\/28\nadd netSpiders 123.151.148.0\/22\nadd netSpiders 5.10.83.0\/25\nadd netSpiders 157.56.0.0\/14\nadd netSpiders 208.115.111.64\/28\nadd netSpiders 54.144.0.0\/12\nadd netSpiders 198.27.64.0\/18\nadd netSpiders 54.224.0.0\/12\nadd netSpiders 54.160.0.0\/12\nadd netSpiders 157.54.0.0\/15<\/code><\/pre>\n\n\n\n
Which will store all sets in the save file.  To restore, it's just the reverse.<\/p>\n\n\n\n
# ipset restore < ipset.save<\/code><\/pre>\n\n\n\n
One thing to bare in mind is that iptables may fail to start if you restore rules that reference any ipsets that don't exist.<\/p>\n\n\n\n
iptables v1.4.21: Set netSpiders doesn't exist.<\/code><\/pre>\n\n\n\n
Startup\/Shutdown.<\/h3>\n\n\n\n
At the time of writing this, ipset does not support systemd.  So I knocked up a script and 2 systemd unit files which seem to do the job very nicely.<\/p>\n\n\n\n
Script<\/a> #### Ipset Startup<\/a> #### Ipset Shutdown<\/a><\/p>\n\n\n\n
Please save the script into \/usr\/sbin\/<\/strong> and both the unit files into \/etc\/systemd\/system\/<\/strong>.  Then enable them.<\/p>\n\n\n\n
 # systemctl enable ipset-save\n # systemctl enable ipset<\/code><\/pre>\n\n\n\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
First thing is to ensure the correct options are set in the kernel. IP sets support the following type of sets: bitmap:ip The bitmap:ip set type uses a memory range, where each bit represents one IP address and can store up to 65535 (B-class network) entries. You can store same size network addresses in this … Continue reading IPTables with IPSets<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[286,285,157,18],"class_list":["post-1642","post","type-post","status-publish","format-standard","hentry","category-gentoo","tag-firewall","tag-ipset","tag-iptables","tag-linux"],"_links":{"self":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts\/1642","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1642"}],"version-history":[{"count":23,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts\/1642\/revisions"}],"predecessor-version":[{"id":2492,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=\/wp\/v2\/posts\/1642\/revisions\/2492"}],"wp:attachment":[{"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1642"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1642"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cdblog.cdstealer.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1642"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}