This document describes the integration process of the ThreatSTOP IP Defense with pfSense 2.1+.

Overview

This document describes how to integrate ThreatSTOP’s Policy and Reporting services with a Netgate pfSense device:

  • Automated retrieval and updates of IP Defense policies
  • Automated collection and upload of log files from the pfSense device to ThreatSTOP’s systems.

System view

The integration is performed by installing a set of scripts provided by ThreatSTOP. After installing the software package, the TSCM will retrieve the list of subnets matching the policy configured via the ThreatSTOP Admin portal and maintain IPFW filtering rules. The software will also upload blocked connection logs to ThreatSTOP’s Portal, for analysis and reporting.

Compatibility

pfSense Compatibility

  • The current version of ThreatSTOP’s pfSense integration is compatible with all pfSense devices running OS versions 2.1.x through 2.4.x. It is compatible with the Netgate appliances, the AWS/Azure images and the community edition.

Current version

The current version of the ThreatSTOP integration is 1.05.

Installation parameters for experienced users

If you have already created a device entry in the portal, and are familiar with the installation procedure, you can access the TSCM parameters below if you access this document from the Portal Device page.

Setting Value
Device ID Retrieved from the device settings page
Policy (Block List) Retrieved from the device settings page
Policy (Allow List) Retrieved from the device settings page

Setup Command

$ php tssetup.php [Block List]  [Allow List]  [Device ID]

Prerequisites

pfSense

  • Firewall table size

  • It is recommended that you save the current configuration before applying ThreatSTOP. This can be done from the Diagnostics: Backup/Restore page of the webConfigurator.

  • If you are running pfSense on a flash filesystem (or another filesystem that is read only by default) then you should temporarily mount the root filesystem as read/write. This is done by entering the following command in the SSH session:
    mount -uw /
    

Connectivity

To retrieve its configuration and policy, and to upload log data, the TSCM needs the following connectivity:

  • DNS over TCP
    • Hostname: dns.threatstop.com
    • IP Range: 192.124.129.0/24
    • Outbound TCP port 53
  • HTTPS
    • Hostname: logs.threatstop.com
    • IP range: 204.68.99.208/28
    • Outbound TCP port 443

Setup

Installing the integration is performed in 4 steps

  1. Configuring the device settings on the Admin Portal
  2. Download the ThreatSTOP software on the pfSense device
  3. Run the configuration command
  4. Enable the ThreatSTOP ACLs

Step 1 - Portal

During this step, you will create a device entry on the Admin Portal. You will select a device type (pfSense 2.x) and enter the configuration settings.

To create a pfSense device entry:

  • Log into the Admin Portal with your ThreatSTOP account
  • Browse to the Device page and click Add Device
  • Select the ASA model:
    • Type: IP Defense
    • Manufacturer: pfSense
    • Model: pfSense 2.x
    • Integration type: On device

The Admin Portal will display a form to enter the device settings described below and the links to retrieve the TSCM image.

  • Nickname: this is a mnemonic name used to identify the device. It can be set to any string (A-Z, 0-9, - and _). If you create multiple device entries, each entry must have a unique nickname. The Nickname will be used to identify the device on the TSCM and in the Reporting user interface.

  • Policy: select a pre-defined policy or a customized policy. It must be an IP Defense Policy.

  • IP Type: Access to the ThreatSTOP services is controlled in part using an ACL allowing the device IP to connect. If your device has a static public IP address (the most common case), select static. If your device has a dynamic public IP address, the ThreatSTOP services can lookup the IP address using a DNS fully-qualified name (FQDN).

  • Public IP address: In static mode, this is the public IP address of the device. It is possible to configure multiple device entries with the same public IP address.

  • Domain name: In Dynamic mode, this is a DNS FQDN which must be kept up-to-date as an A record pointing to the device’s dynamic IP.

  • Internal IP address: This is the internal address of the device. The TSCM will communicate with the ASA via SSH using this IP address. Note: Authentication credentials are documented below.

  • Note: An optional field to store a note of your choice about the device - location, identifiers, model…

Upon saving the form, a device entry will be created in ThreatSTOP’s cloud.

Step 2: Download software

After creating the device entry, the next step is the download using FTP and installation of the TSCM image.

The download link is listed in the Step 2 section, as shown in this image.

  • Click on the Copy Download Link to copy the link to your clipboard
  • Using ssh, login as root to your pfSense device and type the following commands
    cd /root
    curl -LO <download link>
    tar zxvf ts-pfsense.tar.gz
    
  • For your security: after downloading the file, we encourage you to validate its SHA 256 checksum. Compute it as shown below and compare it to the checksum in the Portal.
    $ shasum -a 256 <filename>
    

Step 3: Install and setup

  • To install and setup the ThreatSTOP integration, run the following command:
$ cd ts-pfsense; php tssetup.php [Block List]  [Allow List]  [Device ID]

The software will install automated jobs in the crontab to perform updates and upload logs. It will also add a new page to pfSense’s webConfiguration, under the Firewall menu. Finally, it will retrieve the current version of your policy data.

Sample session:

[2.4.3-RELEASE][admin@pfSense.localdomain]/root: cd ts-pfsense
[2.4.3-RELEASE][admin@pfSense.localdomain]/root/ts-pfsense: php tssetup.php
usage: php tssetup.php blocklist allowlist tdid
[2.4.3-RELEASE][admin@pfSense.localdomain]/root/ts-pfsense: php tssetup.php basic.threatstop.local dns.threatstop.local tdid_12345678
ThreatSTOP setup :
  pfSense version 2.4.3
  ThreatSTOP IP Firewall version 1.05

Adding ThreatSTOP to firewall menu...
Confirming periodic list updates are set...
crontab: no crontab for root
Installation complete.
 Directory '/var/db/aliastables' does not exist. Creating...Done

Getting block and allow lists. Wait please...
ThreatSTOP lists retrieved.
You can now enable ThreatSTOP in the pfSense webConfigurator (Firewall menu).

Step 4: Enable the integration

The final step will enable the integration using the pfSense webConfiguration.

  • Open a web browser and login into the device’s webConfigurator GUI.
  • If not already done, click on the System > Advanced menu then select Nat/Firewall. Set Firewall Maximum Table Entries to at least 400,000.
  • Open the Firewall > ThreatSTOP menu
  • Click the Enable button
  • Confirm that the installation completed ok by checking the output of the setup now displayed on the page.

  • From this point on, the TSCM will retrieve policy data (IP subnets) and update the IPFW entries, every hour.

  • To check the IPFW configuration, run
    [2.4.3-RELEASE][admin@pfSense.localdomain]/root/ts-pfsense: pfctl -sr | grep -i threatstop
    pass in log quick on em0 inet from any to <ThreatSTOP_allow> flags S/SA keep state label "USER_RULE: ThreatSTOP_incoming_lan"
    block drop in log quick on em0 inet from any to <ThreatSTOP_block> label "USER_RULE: ThreatSTOP_incoming_lan"
    pass in log quick on em1 reply-to (em1 172.21.70.1) inet from <ThreatSTOP_allow> to any flags S/SA keep state label "USER_RULE: ThreatSTOP_incoming_wan"
    block drop in log quick on em1 reply-to (em1 172.21.70.1) inet from <ThreatSTOP_block> to any label "USER_RULE: ThreatSTOP_incoming_wan"
    

Logging and Reporting

If log upload is enabled, the TSCM will now upload logs every 15 minutes, as long as there were connections blocked by the policy since the last upload. The logs can be analyzed in the IP Defense Reports 15 minutes after they’ve been uploaded.

Additional considerations

Other operations

Configuration changes

  • To change the policy applied to the device:
    • Update the policy assigned to the device in the Admin Portal
    • Wait 15 minutes for the changes to propagate to ThreatSTOP’s policy service
    • Re-run the tssetup.php command with the updated block and allow policy names.
  • To view the networks contains in the current policy:
    • Login into pfSense’s webConfiguration
    • Open the Diagnostics > Tables menu
    • Select the ThreatSTOP_block or ThreatSTOP_allow to view the subnet list.
  • To disable the ThreatSTOP policy:
    • Login into pfSense’s webConfiguration
    • Open the Firewall > ThreatSTOP menu
    • Click disable
  • To upgrade the ThreatSTOP software:
    • Download the latest version
    • Perform the setup command

Uninstall steps

  • To uninstall the ThreatSTOP integration:
    • Login into pfSense’s webConfiguration
    • Open the Firewall > ThreatSTOP menu
    • Click disable
    • Click Remove ThreatSTOP
    • You can reinstall the software by re-running the setup procedure

Additional information

Troubleshooting

  • Failed to retrieve policy error message
    • If you receive this error message while configurating the ThreatSTOP software, check the connectivity to the ThreatSTOP DNS service described above.
    • If this is a new policy or device, wait 15 minutes for the configuration to propagate.
    • If the issue persists, please contact ThreatSTOP support.
  • There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [18]: table persist file “/etc/bogonsv6”
    • If this error is displayed after enabling ThreatSTOP in the webConfiguration, please make sure that you have increased the firewall table maximum size.
    • See for additional information.

Version history

  • TSCM
Version Release Date Notes
1.05 2018-06-07 Support for 2.4
Error handling and UI enhancements