iptables devices (On Device CLI)

Netfilter’s iptables & ipset

This document goes over ThreatSTOP IP Defense installation integration via Netfilter’s iptables & ipset on a Linux based machine running Debian, or Debian based varients like Ubuntu, CentOS, or RHEL with epel support.

While it is technically possible to use just iptables, we find its orders of magnitude more efficient to use ipset to manage large blocklists.

Current Version

How to check your version

With installs after Ver. 3.00, you can use native package manager commands to find installed version.

admin@linux_server:~$ tsadmin version
5.03-01

If your installation is below v3.00 you will see a No such file or directory response.

Quick settings

curl -fsSL https://debian.threatstop.com/GPG-KEY.asc | sudo apt-key add -
echo "deb https://debian.threatstop.com/ThreatSTOP/ xenial main # xenial #" > /tmp/threatstop.list;
sudo cp /tmp/threatstop.list /etc/apt/sources.list.d/
sudo apt-get update
sudo apt-get install ts-iptables

$ tsadmin add --device_id=[Device ID] --allow_list=[allow list] --block_list=[block list] **WARNING - THIS IS NOT A VALID COMPLETE COMMAND**

Required portal settings during installation.

Please make note of the following settings while going through the CLI installation prompts.

Compatibility

We have tested compatibility with the following:

• May require vault package repository

Install Methods

On Device CLI Install

Installing On-Device via CLI, which this document covers, will have you run through a series of prompts asking for setting particular to your install. The following sections will cover the settings in detail.

On Device Install via Web Automation

Installing On-Device with Web Automation allows you to easily configure settings on our Portal web interface. After configuring the settings on our portal, you will run a command on the device to pull down the configuration. From that point forward your device will sync its configuration updates made on the portal.

Placement of device in network topology

If you are installing this device in an environment that already has a firewall/router it is preferable to install the ThreatSTOP device “inside” the firewall/router if it is doing NAT to track down infected machines on your network. Otherwise the logs will only see the single IP from the next hop instead of the true source node’s IP.

Router mode

If you have setup your device as a router, we will ask for an interface and corresponding interface-type. In router mode this means of the external / edge interface (it is more efficient to drop traffic on the outset).

Bridge mode

If you have setup your device as a network bridge it is important to make sure you load the br_netfilter module. You can do it temporarly on demand by running sudo modprobe br_netfilter. However, we have included an advanced menu item to permanently add it to a modules startup config file. Make sure you answer ‘Y’ - yes to the advanced prompt, or add --load_br_netfilter to the tsadmin command while installing.

More information about bridge netfilter available here.

Inside Bridge Example

Outside Bridge Example

To ensure your bridge device only needs inbound & local firewall rule sets, test traffic to a known test address in both directions. The following logging demonstrates traffic being dropped by the firewall rule-set in both directions by the TS-inbound firewall rule set.

# to get a list of configured bad hosts
admin@linux_server:~$ sudo ipset -L|tail
31.31.23.168
31.31.168.232
# check logs to confirm physical interface direction and that the ThreatSTOP rule matches in both directions
cat /var/log/threatstop/threatstop.log
Jan 18 18:26:24 Linux kernel: [78090.058913] [TS-BlockAddr-17-D] IN=br0 OUT=br0 PHYSIN=eth2 PHYSOUT=eth1 MAC=00:0c:29:80:75:5d:00:0c:29:bd:1c:96:08:00 SRC=10.0.13.232 DST=31.31.23.168 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=9945 DF PROTO=TCP SPT=53329 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0
Jan 18 18:26:25 Linux kernel: [78090.753437] [TS-BlockAddr-15-D] IN=br0 OUT=br0 PHYSIN=eth1 PHYSOUT=eth2 MAC=00:0c:29:bd:1c:96:f0:1c:2d:65:bf:cd:08:00 SRC=10.0.70.82 DST=31.31.168.232 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=54502 DF PROTO=TCP SPT=55964 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0

Positioning firewall rules

We automatically place our ruleset above all others so we block unwanted traffic as early as possible.

High Availability / HA / Cluster installs

We recommend you setup each device as a separate device in the portal and go through the on device configuration just as a normal single device. This way they get unique TDID’s and we can later identify which device reported the indicator of compromise hit.

Installation Overview

This document will go over installation and integration of ThreatSTOP directly in Linux via Netfilter’s iptables & ipset. The basic steps are as follows:

1. Add & Configure device in Admin Portal
2. Add our public software repository & key to your device using native Linux management commands
3. Install ThreatSTOP software via apt-get
4. Configure ThreatSTOP service on device
5. Test configuration / logging.

Prerequisites

• Device added & configured via the admin portal.
• SSH access with su privileges to the Linux host.
• Current active ThreatSTOP account.

Minimum hardware requirements

Whether you are installing Linux directly onto bare metal hardware or a virtual machine, we recommend the following minimum specifications.

• Memory 1GB for testing / 512MB or less can be used unless you are planning on using lots of features.
• Hard Drive 4GB should more than cover both the OS and the device integration.

ThreatSTOP Communication Overview

Connectivity

To retrieve its configuration and policy, and to upload log data, the device needs the following connectivity:

• DNS over TCP - Policy service
  • Hostname: ts-dns.threatstop.com
  • IP Range: 192.124.129.0/24
  • Outbound TCP port 53 or 5353
• DNS over TLS - Configuration service
  • Hostname: ts-ctp.threatstop.com
  • IP Range: 204.68.97.208/28
  • Outbound TCP port 5353
• HTTPS - Log service
  • Hostname: logs.threatstop.com
  • IP range: 204.68.99.208/28
  • Outbound TCP port 443
  • Direct Connection or via Proxy

Setup

Step 1: Portal device configuration

During this step, you will create a device entry on the Admin Portal. You will select a device type and enter the configuration settings. A minimum configuration only requires a handful of settings but optional, advanced settings are also available.

To create a device entry:

• Log into the Admin Portal with your ThreatSTOP account
• Browse to the Device page and click Add Device
• Select the model:
  • Type: IP Defense
  • Manufacturer: Linux
  • Model: iptables (Ubuntu)
  • Integration Type: On device

The Admin Portal will display a form to enter the device settings described below.

• Nickname: this is a mnemonic name used to identify the device. It can be set to any string (A-Z, 0-9, - and _). If you create multiple device entries, each entry must have a unique nickname. The Nickname will be used to identify the device on the device and in the Reporting user interface.

• Policy: select a pre-defined policy or a customized policy. It must be an IP Defense Policy.

• IP Type: Access to the ThreatSTOP services is controlled in part using an ACL allowing the device IP to connect. If your device has a static public IP address (the most common case), select static. If your device has a dynamic public IP address, the ThreatSTOP services can lookup the IP address using a DNS fully-qualified name (FQDN).

• Public IP address: In static mode, this is the public IP address of the device. It is possible to configure multiple device entries with the same public IP address.

• Note: An optional field to store a note of your choice about the device - location, identifiers, model…

Step 2. Setup software package repository

Adding the repository key

You’ll need to add the repository key as we sign the debian packages for safety and security.

# download the threatstop public repo key
wget https://debian.threatstop.com/GPG-KEY.asc
# compare the hash
shasum -a 256 GPG-KEY.asc
# add verified key to aptitude
sudo apt-key add GPG-KEY.asc

Adding the repository

Log into your device via SSH and start a configuration session by typing configure.

Type or copy & paste the following commands to add the ThreatSTOP software repository.

echo "deb https://debian.threatstop.com/ThreatSTOP/ xenial main # xenial #" > /tmp/threatstop.list;
sudo cp /tmp/threatstop.list /etc/apt/sources.list.d/;
sudo apt-get update;

Step 3. Install ThreatSTOP software via apt-get

The ThreatSTOP on device software will download via aptitude package manager. Depending on the OS release you may have other dependancies as well. We try to host all required dependancies such as dnsutils. However, if you are unable to install our service due to this, please contact support.

Install by running:

sudo apt-get update
sudo apt-get install ts-iptables

After installation you will be able to configure the device via the tsadmin add command.

Step 4. Configure ThreatSTOP service on device

This document covers the CLI method of installation using the tsadmin add command. This method will guide you through setting up the device via a series of prompts described in this document.

Configure

1. SSH into the Linux device.

2. To begin configuring the device, run the following command:

   $ tsadmin add --device_id=[Device ID] --allow_list=[allow list] --block_list=[block list] **WARNING - THIS IS NOT A VALID COMPLETE COMMAND**
   

   Note: During tsadmin prompts, values in the square brackets are defaults and can be used by hitting return

3. The CLI installation will prompt you for the following settings. Please have them ready along with device id, block & allow lists available here.

   • Device ID: This is the device id or TDID given in the quick settings section or referenced in the portal.

   • TSPrefix: This is a prefix we prepend to allow / block ipset groups. We recommend keeping this extremely short (no more than 29 characters).

   • Block list: This is the ThreatSTOP block list given in the quick settings section or referenced in the portal.

   • Allow list: This is the ThreatSTOP allow list given in the quick settings section or referenced in the portal.

   • Maximum Policy Size: Option limit on the number of entries in the policy. If the policy becomes larger than this setting, the device will truncate it down to the Maximum Policy Size.

   Advanced Settings

   • DNS Port: The device uses TCP Port 53 (outbound connections) to retrieve policy data. If this port is blocked or filtered (for example, networks using a DNS Application Layer Gateway), use this setting to switch to TCP Port 5353.

   • Load br_netfilter: This (yes/no) prompt will allow tell the installation to create a permanent configuration file to load br_netfilter so that bridged traffic get’s processed via the systems iptables rulesets.

   • Logopts: This field is used to set additional iptables logging rules see iptables logging options for more details.

   • Logopts Direction: This sets what direction (inbound, outbound, or both) we will setup the iptables “logopts” logging options.

4. After the initial configuration questions you will prompted to “CONFIRM THREATSTOP INSTALL”. This is your last chance to confirm settings and proceed with the actual install.

5. Finally, you will be asked if you wish to download the policy now. It is safe to do so if it’s been more than 15 minutes since you’ve added the device in the portal. If you choose not to download the policy now, it will be automatically downloaded in one hour via the crontab entry.

Proceed to Step 5. Testing your configuration.

Step 5. Test configuration / logging

To test your integration to ensure proper functionality we will attempt to ping a known test address added to most of our IP Defense policies (bad.threatstop.com / 64.87.3.133).

# 64.87.3.133 is ThreatSTOP's sample threat indicator bad.threatstop.com
ping 64.87.3.133
# after about 3-5 seconds quit the command by hitting CTRL+C

You should see traffic being properly filtered as shown below.

admin@linux_server:~$ ping 64.87.3.133
PING 64.87.3.133 (64.87.3.133) 56(84) bytes of data.
^C
--- 64.87.3.133 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2006ms

Now we will check to ensure we are logging that filtered traffic.

cat /var/log/user/threatstop.log

You should see some log lines reporting blocks similar to those shown below.

Jan 15 21:26:05 Linux kernel: [TS-Blockaddr-13-D]IN=eth0 OUT= MAC=24:a4:3c:3d:20:e2:00:0c:XX:XX:XX SRC=64.87.3.133 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=57 ID=56423 PROTO=ICMP TYPE=0 CODE=0 ID=5925 SEQ=1
Jan 15 21:26:06 Linux kernel: [TS-Blockaddr-13-D]IN=eth0 OUT= MAC=24:a4:3c:3d:20:e2:00:0c:XX:XX:XX SRC=64.87.3.133 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=57 ID=56606 PROTO=ICMP TYPE=0 CODE=0 ID=5925 SEQ=2
Jan 15 21:26:07 Linux kernel: [TS-Blockaddr-13-D]IN=eth0 OUT= MAC=24:a4:3c:3d:20:e2:00:0c:XX:XX:XX SRC=64.87.3.133 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=57 ID=56670 PROTO=ICMP TYPE=0 CODE=0 ID=5925 SEQ=3

If you see the logging lines and blocked traffic, Congratulations, you’ve successfully installed IP Defense! Your first logs should be visible on the portal in 15min to 2 hours depending on the cycle.

You can manually send logs by doing the following. Depending on your installed OS the output of the commands below may differ, but this should still serve as a general guide.

admin@linux_server:~$ tsadmin logs

tsadmin sub commands

tsadmin has several sub commands to perform various functions outlined below. Sub commands are interpreted differently by the tsadmin & do not have a -- before them.

• add The add command install & configures the ThreatSTOP service on the device.
• remove This command removes the ThreatSTOP integration from the device.
• update This command forces the device to update its policy as well as check for new settings if web automation installation is used.
• show This command shows the configured settings.
• version This command shows the current version of package.
• logs This command forces log rotation & sends logs as well as restarting logging facilities if needed.

tsadmin add cli sub parameters

The following tsadmin parameters can be added inline to seed the installation.

• advanced Run advanced prompts
• allow_list ThreatSTOP Allow list
• auto_key Web Automation configuration key
• block_list ThreatSTOP Block list
• device_id ThreatSTOP device ID (TDID). i.e. --device_id=<span class='rtv-tdid'>[Device ID]</span>
• dns_server DNS server
• load_br_netfilter Load Linux kernel bridge netfilter module and add to startup. Valid choices: (y, n)
• logopts Extra iptables log options (i.e. –log-uid, –log-level 6, etc…)
• logopts_direction Traffic direction for extra iptables log options. Valid choices: (in, out, both)
• maxpolicysize Maximum size of block list items (%20 for allow list).
• port DNS port for upstream queries
• silent Suppress output & auto accept all prompts. Useful when automating cli installs.
• skip_checkpolicy Skip validating your ThreatSTOP policy exists and reachable. Useful in offsite installations.
• skip_checkvalidip Skip validating your public IP is registered with ThreatSTOP. Useful in offsite installations.
• take_defaults Auto accept all prompts with confirmation countdown. Useful when automating cli installs.
• tsprefix Prefix for iptables / IPset rules (1-19 chars)
• type Installation type (auto - web automation, manual - via the CLI prompts). Defaults to manual.

Troubleshooting

To get a list of configured settings run:

tsadmin show

To get a full list of available command line parameters on the command line type:

tsadmin --help

If you are not able to download the policy, run the following:

admin@linux_server# wget -qO - http://logs.threatstop.com/cgi-bin/validip.pl
Your IP address: 1.2.3.4
Address is in the list of authorized hosts

This will tell you if your public IP has been allowed to access the policy. If you are unable to get a response it means you need to update the public IP on the devices section of the admin portal.

Version Changelog

ts-iptables (5.00-01) trusty; urgency=low
 * GS-2013 : Major Refactor with SDC
           : Added web automation
           : Added enhanced validation and CLI interface
 -- ThreatSTOP Support <support@threatstop.com>  Thu, 16 May 2019 10:20:00 -0700
ts-iptables (4.15-02) trusty; urgency=low

  * GS-1100 : support new and old style Net::DNS
  * GS-1198 : move log file to threatstop-controlled directory

 -- ThreatSTOP Support <support@threatstop.com>  Fri, 4 Nov 2016 18:35:34 -0700
ts-iptables (4.14-00) trusty; urgency=low

  * AZ-84   : provide a script that allow for changing of the block and allow policies
  * GS-623  : don't skip future log file name changes, just because the config is already there
  * GS-1068 : do not allow installation directory to be configurable
              (debian packaging controls this location now)

iptables logging options

The following options are typically available on iptables 1.4+ Ref. Please refer to your specific iptables manual for exact details. The following is here as a general reference:

• --log-tcp-sequence Log TCP sequence numbers. This is a security risk if the log is readable by users.
• --log-tcp-options Log options from the TCP packet header.
• --log-level 4 This is the standard syslog levels. 4 is warning. You can use number from the range 0 through 7. 0 is emergency and 7 is debug.
• --log-ip-options Log options from the IP packet header.
• --log-uid Log the userid of the process which generated the packet.

Uninstalling ThreatSTOP

Remove ThreatSTOP service integration from device

To uninstall the service you can run:

$ tsadmin remove
    ThreatSTOP ts-iptables remove starting @ Wed May 29 22:40:20 2019

Are you sure you want to uninstall ThreatSTOP? (y/n) y

The same script get triggered when removing the software via aptitude by running sudo apt-get remove ts-iptables. In both cases your config file will be left behind in /opt/threatstop/threatstop.conf incase you wish to re-install.

Restore previous configuration

We create a backup of the configuration prior to installing our service located at /opt/threatstop/pre-threatstop_ipset & /opt/threatstop/pre-threatstop_iptables.

You should be able remove all threatstop related rules / configuration with tsadmin remove. However in the event something goes wrong you can take the following steps to manually remove the rules.

The most common installation of iptables & ipset loads the rules into a kernel extension. For this reason we can’t just delete the rules without unloading them from the Linux kernel first. The general method of removing rules is as follows:

1. flush ipset hashsets
2. remove rules using those hashsets from iptables
3. destroy the hashsets

manually removing threatstop rules

1. Uninstall ts-iptables package. This ensures the threatstop_iptables_save & threatstop_iptables_restore scripts don’t save/load previous rulesets on startup.
2. Reboot (which will not load old rules), or if rebooting is not possible.
3. Gather a list of ipset hashset names to remove by running sudo ipset -L -name. ThreatSTOP’s naming convention ends with (allowaddr, allownet, blockaddr, blocknet). We also create temporary rulesets ending in (allowaddrnew, allownetnew, blockaddrnew, blocknetnew) prior to swapping new rules in place.
4. For each rule name run sudo ipset flush <rulename>
5. For each rule name identify and remove iptables rules using that rule. sudo iptables -L ThreatSTOP --line-numbers. You can simply run sudo iptables -D <chain name> <rule number>.
6. For each rule name run sudo ipset destroy <rulename>

This should complete the uninstallation of threatstop rules.

References

• General iptables guide
• Info regarding Netfilter’s bridge kernel module
• Using iptables rules in bridge mode
• iptables manual, along with some log option definitions
• Information regarding ipset’s maxelem vs hashsize