This document describes how to integrate ThreatSTOP’s IP Defense with a Netgate pfSense device:
- Automatic & persistent configuration via web portal. You set policy & integration configuration on our admin portal, then the settings automatically propagate on the device. Think of this as unified central management across device brands, makes and models.
- Automated retrieval and updates of IP Defense policies
- Automated collection and upload of log files
We have tested compatibility with the latest pfSense version (2.5.0) at the time of the release of this document:
Unfortunately, Netgate has deprecated 2.4.x and prior releases. While recently testing our previous (2.00) package on 2.4.x,
pkg says it needs to upgrade which breaks the package manager leading to a
libarchive.so.7 not found error. See example below.
[2.4.5-RELEASE][admin@pfSense.localdomain]/root: pkg install ts-pfsense-2.01-10.txz pkg: Warning: Major OS version upgrade detected. Running "pkg bootstrap -f" recommended Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. New version of pkg detected; it needs to be installed first. The following 1 package(s) will be affected (of 0 checked): Installed packages to be UPGRADED: pkg: 1.15.6 -> 1.16.1 [pfSense] Number of packages to be upgraded: 1 The process will require 16 MiB more space. 7 MiB to be downloaded. Proceed with this action? [y/N]: n
If you have a current 2.4 or older pfSense that is currently working, it should continue to work. However we still recommend upgrading since updating any package will likely lead to a broken system.
|Netgate||pfSense v2.4.x and prior||Deprecated, See this and this|
While we do offer manual support for older versions of pfSense older than 2.5.0 we strongly urge you to upgrade as there are many other security related concerns along with FreeBSD versions being “End of Life” and out of support.
ThreatSTOP Data Flow Diagram
On Device CLI Install
Installing On-Device via SSH CLI, which is covered here.
On Device Install via Web Automation
Installing directly on your device via Web Automation, which allows you to configure settings on our web interface and have them automatically update on the device after initial installation. This method will require you to run an initial abbreviated command on the device to link up to our services. From that point forward your device will stay in sync with configuration updates you make on the portal. For more information regarding Web Automation click here.
Quick settings are provided below for expert installers who have already read through the documentation and understand what they are doing.
curl -LO https://ftp.threatstop.com/pub/ts-pfsense-2.01-10.txz && pkg install ./ts-pfsense-2.01-10.txz && rehash # now run the tsadmin add --type auto command directly below
$ tsadmin add --type auto --device_id=[Device ID] --auto_key=[Device Key] **WARNING - THIS IS NOT A VALID COMPLETE COMMAND**
Network Bridge Considerations
If you have setup your device as a network bridge it is important to make sure you enable the
Packet filter on bridge interface checkbox see this guide. Aside from bridge specific setup, the ThreatSTOP integration steps are the same for bridges as they are for routers.
More information about Netgate setting up a transparent bridge available here.
Inside Bridge Example
Outside Bridge Example
Placement of device in network topology
If you are installing this device in an environment that already has a firewall/router it is preferable to install the ThreatSTOP device “inside” the firewall/router if it is doing NAT to track down infected machines on your network. Otherwise the logs will only see the single IP address from the next hop instead of the true source node’s IP address.
Please make note of the following settings while going through the installation prompts.
|Device ID||Retrieved from the device settings page|
|Auto Key||Retrieved from the device settings page|
This document will go over installation and integration of ThreatSTOP via web automation directly on to your pfSense device. The basic steps are as follows:
- Add & Configure device in Admin Portal
- Install ThreatSTOP software
- Configure ThreatSTOP service on device
- Test configuration / logging.
- Device added & configured via the admin portal.
- Firewall Device setup as either a router or a network bridge.
- SSH access to the device.
- Current active ThreatSTOP account.
- For web automation you will need to have your time synced
- Firewall table size set high enough to accommodate your policy
Warning: Prior to installing the ThreatSTOP package, you will need to increase the Firewall Maximum Table Entries size to at least 400,000 entries in System > Advanced, Firewall & NAT.
- If you are running pfSense on a flash filesystem (or another filesystem that is read only by default), you will have to temporarily mount the root filesystem as read/write. This is done by entering the following command in the SSH session:
mount -uw /
To retrieve its configuration and policy, and to upload log data, the device needs the following connectivity:
- DNS over TCP - Policy service
- Hostname: dns.threatstop.com
- IP Address Range: 18.104.22.168/24
- Outbound TCP port 53 or 5353
- DNS over TLS - Configuration service
- Hostname: ts-ctp.threatstop.com
- IP Address Range: 22.214.171.124/28
- Outbound TCP port 5353
- HTTPS - Log service
- Hostname: logs.threatstop.com
- IP Address Range: 126.96.36.199/28
- Outbound TCP port 443
- Direct Connection or via Proxy
- Hostname: time.nist.gov
- Outbound UDP port 123
Step 1 - Portal device configuration
During this step, you will create a device entry on the Admin Portal. You will select a device type and enter the configuration settings. A minimum configuration only requires a handful of settings but optional, advanced settings are also available.
To create a device entry:
- Log into the Admin Portal with your ThreatSTOP account
- Browse to the Device page and click Add Device
- Select the model:
- Type: IP Defense
- Manufacturer: Netgate
- Model: pfSense
- Integration Type: On device (Web Automation)
The Admin Portal will display a form to enter the device settings described below.
Nickname: this is a mnemonic name used to identify the device. It can be set to any string (A-Z, 0-9, - and _). If you create multiple device entries, each entry must have a unique nickname. The Nickname will be used to identify the device on the device and in the Reporting user interface.
Policy: select a pre-defined policy or a customized policy. It must be an IP Defense Policy.
IP Type: Access to the ThreatSTOP services is controlled in part using an ACL allowing the device IP to connect. If your device has a static public IP address (the most common case), select static. If your device has a dynamic public IP address, the ThreatSTOP services can lookup the IP address using a DNS fully-qualified name (FQDN).
Public IP address: In static mode, this is the public IP address of the device. It is possible to configure multiple device entries with the same public IP address.
Note: An optional field to store a note of your choice about the device - location, identifiers, model…
Step 2 - Download Software
The download link is listed in the Step 2 section, as shown in this image.
- Click on the Copy Download Link to copy the link to your clipboard
- Using ssh, login as root to your pfSense device and type the following commands
# first we will download the software from out public repo curl -LO https://ftp.threatstop.com/pub/ts-pfsense-2.01-10.txz # For your security: after downloading the file, we encourage you to validate its SHA 256 checksum. Compute it as shown below and compare it to the checksum in the Portal. shasum -a 256 <filename> # now we install via pkg facility pkg install ./ts-pfsense-2.01-10.txz # you will be prompted with some dependencies if they are not already on the system. # next try running tsadmin, if you get error, you may have to log out / back in or run `rehash` to reindex the new applications # the following is an example to show we have tsadmin in path and ready to install. [2.4.4-RELEASE][root@pfSense.localdomain]/root: tsadmin version ThreatSTOP ts-pfsense version starting @ Wed Oct 2 22:34:35 2019 ts-pfsense Version: 2.00
Step 3 - Configuration
SSH into the pfSense device with an admin account. If you are not root make sure you run commands prefixed with
To begin configuring the device, run the following command:
$ tsadmin add --type=auto --device_id=[Device ID] --auto_key=[Device Key]Note: During this step, the tsadmin will attempt build a secure channel to pull down all settings to configure itself.
The CLI installation will pull down the following configuration settings for you as configured on the admin portal.
Device ID: This is the device identification used to associate logs and settings to specific device.
Block list: This is the ThreatSTOP block list given in the quick settings section or referenced in the portal.
Allow list: This is the ThreatSTOP allow list given in the quick settings section or referenced in the portal.
Maximum Policy Size: Option limit on the number of entries in the policy. If the policy becomes larger than this setting, the device will truncate it down to the Maximum Policy Size.
- DNS Port: The device uses TCP Port 53 (outbound connections) to retrieve policy data. If this port is blocked or filtered (for example, networks using a DNS Application Layer Gateway), use this setting to switch to TCP Port 5353.
Next, you will be asked to confirm ThreatSTOP install. After you accept, the software will stage installation files and attempt to fetch the latest policy.
Once the ThreatSTOP software is installed you can enable the policy by going to your devices web interface, then selecting Firewall > ThreatSTOP and sliding the disabled button into the enabled state.
Now, proceed to “Testing your Configuration” step below.
Step 4 - Testing Configuration
To ensure your device is properly blocking and logging traffic, test traffic to a known test address in both directions. The following logging demonstrates traffic being dropped by the firewall rule-set in both directions by the TS-inbound firewall rule set.
# first we test to see if bad.threatstop.com's IP address is being blocked. This is a safe IOC to use for testing. admin@pfSense# dig +short bad.threatstop.com 188.8.131.52 # next, we verify it is in the ThreatSTOP block list. admin@pfSense# grep 184.108.40.206 /var/db/aliastables/ThreatSTOP_block.txt 220.127.116.11 # The following step is best done on a client machine routing through the firewall. # We'll try to curl or visit the address http://bad.threatstop.com to see if we can generate a block and log event. curl bad.threatstop.com # or attempt to visit with a web browser # check logs to confirm physical interface direction and that the ThreatSTOP rule matches in both directions admin@pfSense# clog /var/log/filter.log | grep 18.104.22.168 Sep 20 23:37:41 pfSense filterlog: 6,,,1000000104,em1,match,block,out,4,0x0,,64,0,0,DF,6,tcp,52,22.214.171.124,192.168.56.1,80,57232,0,RA,1848334924,822794810,513,,nop;nop;TS
If you see the logged test blocks, Congratulations, you’ve now protected your network with a world class security threat feed!
Be sure to read our Portal Guide to get the most out of our advanced reporting features.
- To check the IPFW configuration, run
[2.4.3-RELEASE][admin@pfSense.localdomain]/root/ts-pfsense: pfctl -sr | grep -i threatstop pass in log quick on em0 inet from any to <ThreatSTOP_allow> flags S/SA keep state label "USER_RULE: ThreatSTOP_incoming_lan" block drop in log quick on em0 inet from any to <ThreatSTOP_block> label "USER_RULE: ThreatSTOP_incoming_lan" pass in log quick on em1 reply-to (em1 172.21.70.1) inet from <ThreatSTOP_allow> to any flags S/SA keep state label "USER_RULE: ThreatSTOP_incoming_wan" block drop in log quick on em1 reply-to (em1 172.21.70.1) inet from <ThreatSTOP_block> to any label "USER_RULE: ThreatSTOP_incoming_wan"
Changing the policy
* Update the policy assigned to the device in the Admin Portal * Wait 15 minutes for the changes to propagate to ThreatSTOP's policy service * The web automation periodic update should pick up new changes and apply them automatically. Please check the Firewall > ThreatSTOP page to verify new policy details.
View the networks contains in the current policy
* Login into pfSense's webConfiguration * Open the **Diagnostics > Tables** menu * Select the **ThreatSTOP_block** or **ThreatSTOP_allow** to view the subnet list.
Disable the ThreatSTOP policy
* Login into pfSense's webConfiguration * Open the **Firewall > ThreatSTOP** menu * Click **disable**
- log into the web interface and go to Firewalls > ThreatSTOP
- from this status page you’ll be able to disable, then remove the integration
- next remove the previous integration files (you can search your filesystem for tssetup.php if you forgot where you installed it.)
- you can now proceed installing the new version.
How to check your version
$ tsadmin version 2.01-10
Builds older that 2.x have their version listed in a
VERSION file in the code directory. Another quick way of identifying your version is
tsadmin command was introduced in v2.00. If your build does not have this command it is v1.x.
tsadmin has several sub commands to perform various functions outlined below. Sub commands are interpreted differently by the tsadmin & do not have a
-- before them.
- add The add command install & configures the ThreatSTOP service on the device.
- remove This command removes the ThreatSTOP integration from the device.
- update This command forces the device to update it’s policy as well as check for new settings if web automation installation is used.
- show This command shows the configured settings.
- version This command shows the current version of installation package.
- logs This command forces logs to be sent to ThreatSTOP as well as restarting logging facilities if needed.
tsadmin add sub parameters
tsadmin add sub parameters can be added inline to seed the installation. If you are installing via web automation you will only be using auto_key and device_id.
- advanced Run advanced prompts. i.e.
- allow_list ThreatSTOP Allow list. i.e.
- auto_key Web Automation configuration key i.e.
- block_list ThreatSTOP Block list i.e.
- device_id ThreatSTOP device ID (TDID). i.e.
- dns_server DNS server
- maxpolicysize Maximum size of block list items (%20 for allow list).
- port DNS port for upstream queries
- silent Suppress output & auto accept all prompts. Useful when automating cli installs. i.e.
- skip_checkpolicy Skip validating your ThreatSTOP policy exists and reachable. Useful in offsite installations.
- skip_checkvalidip Skip validating your public IP address is registered with ThreatSTOP. Useful in offsite installations.
- take_defaults Auto accept all prompts with confirmation countdown. Useful when automating cli installs. i.e.
- type Installation type (auto - web automation, manual - via the CLI prompts). Defaults to manual
- Failed to retrieve policy error message
- If you receive this error message while configuring the ThreatSTOP software, check the connectivity to the ThreatSTOP DNS service described above.
- If this is a new policy or device, wait 15 minutes for the configuration to propagate.
- If the issue persists, please contact ThreatSTOP support.
- There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table bogonsv6: Cannot allocate memory - The line in question reads : table persist file “/etc/bogonsv6”
To get a list of configured settings run:
To get a full list of available command line parameters on the command line type:
If you are not able to download the policy, run the following:
admin@pfSense# curl http://logs.threatstop.com/cgi-bin/validip.pl Your IP address: 126.96.36.199 Address is in the list of authorized hosts
This will tell you if your public IP address is authorized to access the policy.
|2.01||2021-03-31||Support for 2.5.0|
|2.00||2019-10-07||Support for 2.4.4/ PHP7 / web automation integration / enhanced integration UX|
|1.05||2018-06-07||Support for 2.4
Error handling and UI enhancements
- To uninstall the ThreatSTOP integration:
- Login into pfSense’s webConfiguration
- Open the Firewall > ThreatSTOP menu
- Click disable
- Click Remove ThreatSTOP
- You can reinstall the software by re-running the setup procedure
- You can also remove it by running
tsadmin removewhile logged in via SSH.
We recommend you keep your firewall’s time synced if you intend to use our web automation services. It is required to provide a secure channel of communication to the device for persistent configuration & updates.
Please review and set per your manufacturers recommendation.
Backup previous configuration
It is recommended that you save the current configuration before applying ThreatSTOP. This can be done from the Diagnostics: Backup/Restore page of the webConfigurator.
Restore previous configuration
We create a backup of a few key configuration directories prior to install. Simply go to the ThreatSTOP status page via the web-interface (Firewall=> ThreatSTOP menu) and click the “Remove ThreatSTOP” button after disabling the service. You can also run
tsadmin remove from the command line if logged in via SSH.
If all else fails contact support for assistance.