Fortinet Fortigate integration (TSCM CLI)

Overview

This document describes how to integrate ThreatSTOP’s Policy and Reporting services with a Fortinet Fortigate device:

• Automated retrieval and updates of IP Defense policies from ThreatSTOP’s systems to the Fortigate.
• Automated collection and upload of log files from the Fortigate to ThreatSTOP’s systems.

System view

The integration is performed by a Linux-based virtual machine provided by ThreatSTOP, named ThreatSTOP Centralized Manager (TSCM). After its initial installation, the TSCM will retrieve the list of subnets matching the policy configured via the ThreatSTOP Admin portal and update the Fortigate using an SSH connection. Optionally, the Fortigate can be configured to send the connection log events to the TSCM via syslog and the TSCM will package and upload log files to ThreatSTOP’s Portal, for analysis and reporting.

Fig 1. : Network traffic between ThreatSTOP services, the TSCM and the Fortigate. (click to expand)

Shell configuration

This document provides the steps using the Command Line-based installation of ThreatSTOP. Using the Command Line method, device settings are configured on the command line. The initial configuration and future changes must be done using the TSCM CLI. Changes to the policy assignment must be done on both the portal and TSCM CLI.

For the Web-based installation, please read this document instead.

Compatibility

Fortigate Compatibility

• The current version of TSCM is compatible with all Fortigate devices running FortiOS version 4.x, 5.0-5.6, 6.x, 7.0.x.

• The Maximum Policy Size is the maximum number of ACLs that your device can support. It depends on the model number and version of FortiOS. It is documented by Fortinet in the Fortinet Document Library : search for “Maximum Values:, select your version of FortiOS and look for for the number of Firewall Addresses supported by your model number.

• FortiOS 5.4 introduced new features to the address object, which significantly impacts the performance of ACL updates. It is recommended to limit policies to 2000 ACLs or less on models smaller than FG-1000c when running FortiOS 5.4 and later.

• VDOMs are supported

• The Maximum Policy Size is the maximum number of ACLs than your device can support. It depends on the hardware and memory available on the device. Please contact your Fortinet reseller to find out the capacity of your device.

Current version of TSCM

The current version of the TSCM virtual machine is 1.50. If your TSCM image is older, please download the latest version from the device configuration page in the Admin Portal. You can find out the TSCM version by running

$ tsadmin version

The current version of the Fortigate module is 4.03 (included with TSCM 1.50 images)

Installation parameters for experienced users

If you have already created a device entry in the portal, and are familiar with the installation procedure, you can access the TSCM parameters below if you access this document from the Portal Device page.

Prerequisites

System

The TSCM is delivered as an OVA or VHD image, built using Ubuntu 20.04 as the base Operating System. It is preconfigured with:

• 2 CPUs
• 2 GB of RAM
• 20 GB of disk space

You will need a Hypervisor such as vSphere, ESXi, Virtualbox or Hyper-V to deploy the image.

Connectivity

To retrieve its configuration and policy, and to upload log data, the TSCM needs the following connectivity:

• DNS over TCP - Policy service
  • Hostname: ts-dns.threatstop.com
  • IP Range: 192.124.129.0/24
  • Outbound TCP port 53 or 5353
• HTTPS - Log service
  • Hostname: logs.threatstop.com
  • IP range: 204.68.99.208/28
  • Outbound TCP port 443
  • Direct Connection or via Proxy
• NTP
  • Hostname: ntp.ubuntu.com
  • Outbound UDP port 123

It must also be able to communicate with the Cisco device:

• SSH
  • TCP Port 22
  • From the TSCM to the Fortigate
• Syslog
  • TCP or UDP Port 514
  • From the Fortigate to the TSCM

Fortigate credentials

To perform this installation, you need an account reachable via SSH on the Fortigate as well as the Enable password for the device.

Setup

Integration ThreatSTOP with a Fortigate device using the TSCM CLI is performed in 4 steps:

1. Configuring the device settings on the Admin Portal
2. Downloading and loading the VM image
3. Configure the device settings using the TSCM CLI
4. Apply ACLs and configure log forwarding on the Fortigate

Step 1: Portal

Integration ThreatSTOP with a Fortigate device using Web Automation is performed in 4 steps:

1. Configuring the device settings on the Admin Portal
2. Downloading and loading the VM image
3. Linking the TSCM to the Portal device configuration
4. Apply ACLs and configure log forwarding on the Fortigate

To create a Fortigate device entry:

• Log into the Admin Portal with your ThreatSTOP account
• Browse to the Device page and click Add Device
• Select the Fortigate model:
  • Type: IP Defense
  • Manufacturer: Fortinet
  • Model: Fortigate
  • Integration Type: TSCM

The Admin Portal will display a form to enter the device settings described below and the links to retrieve the TSCM image.

• Nickname: this is a mnemonic name used to identify the device. It can be set to any string (A-Z, 0-9, - and _). If you create multiple device entries, each entry must have a unique nickname. The Nickname will be used to identify the device on the TSCM and in the Reporting user interface.

• Policy: select a pre-defined policy or a customized policy. It must be an IP Defense Policy.

• IP Type: Access to the ThreatSTOP services is controlled in part using an ACL allowing the device IP to connect. If your device has a static public IP address (the most common case), select static. If your device has a dynamic public IP address, the ThreatSTOP services can lookup the IP address using a DNS fully-qualified name (FQDN).

• Public IP address: In static mode, this is the public IP address of the device. It is possible to configure multiple device entries with the same public IP address.

• Domain name: In Dynamic mode, this is a DNS FQDN which must be kept up-to-date as an A record pointing to the device’s dynamic IP.

• Note: An optional field to store a note of your choice about the device - location, identifiers, model…

Upon saving the form, a device entry will be created in ThreatSTOP’s cloud.

Step 2: Download and boot image

After creating the device entry, the next step is the download using FTP and installation of the TSCM image.

You can choose between the OVA format (ESXi/vSphere, VirtualBox, Xen…) and the VHD format (Microsoft Hyper-V).

The download link is listed in the Step 2 section, as shown in this image.

• Click on the Copy Download Link to copy the link to your clipboard
• Use an ftp client of your choice, or a tool such as curl
• For your security: after downloading the file, we encourage you to validate its SHA 256 checksum. Compute it as shown below and compare it to the checksum in the Portal.

  $ shasum -a 256 <filename>
  

• Import the OVA or VHD file in your Hypervisor to create the virtual machine and start it.

Log into the TSCM

The TSCM virtual machine will use DHCP to obtain its IP address. If your Hypervisor doesn’t show the IP address assigned to the virtual machine, you can retrieve it from the console of the TSCM: it is displayed as part of the login prompt.

The virtual machine will be reachable using ssh:

• The default username is: threatstop
• The default password is: threatstop

Step 3: Configure the Device settings using the TSCM CLI

After booting the TSCM and logging in via ssh, the third setup step will link the virtual machine to the device entry created in Step 1.

The TSCM has a configuration utility named tsadmin. A reference for the utility is provided here but we will cover the full installation steps below.

• Obtain the Device ID and Policy zone names from the device configuration or device info pages on the portal

• Login with the threatstop account using ssh

• Validate that the TSCM can reach the Fortigate using SSH

  ssh admin@<Fortigate IP address>
  The authenticity of host '<ip address>' can't be established.
  ECDSA key fingerprint is SHA256:UW05wRgAblpwjfObj4ZklSYfau8PnoE1GXXuSCO5Zfs.
  Are you sure you want to continue connecting (yes/no)?
  

  • Accept the host key
  • Login to validate your password
• Run the following command:

$ tsadmin add --type Fortigate [device nickname]

• The tsadmin command will prompt you for the following device settings.

  • Device ID (tdid): A unique identifier for the device. This setting must be correct for the TSCM to operate normally. This setting can be retrieved from the Device settings or Device Info page on the portal.

  • Device Management IP address: This is the internal address of the device. The TSCM will communicate with the Fortigate via SSH using this IP address.

  • Block list name: the name of the zone associated with the Block list of the policy selected in the portal. This setting can be retrieved from the Device settings or Device Info page on the portal.

  • Allow list name: the name of the zone associated with the Allow list of the policy selected in the portal. This setting can be retrieved from the Device settings or Device Info page on the portal.

  • Enable Virtual Domain support: Must be set to yes if your Fortigate is configured to use the Virtual Domains feature. VDOMs can be used to divide a single FortiGate unit into two or more virtual instances of FortiOS that function as independent FortiGate units. ThreatSTOP ACLs can be assigned to a VDOM.

  • Virtual domain name: If you enabled VDOM support, enter the name of the VDOM to use. VDOM names are case-sensitive.

  • Configure syslog: Set to Yes to automatically configure the Fortigate device to send logs to the TSCM.

  • Max entries per policy: select the maximum number of ACLs (Addresses) supported by your combination of device model and FortiOS. Fortinet’s documentation is linked here.

  • Maximum entries per address group: select the maximum number of Addresses per group documented by Fortinet. Fortinet’s documentation is linked here. The TSCM will split the policy into groups of the specified size.

  • Policy Name prefix: provide a prefix to easily identify ACLs managed by ThreatSTOP’s integration.

  • Trusted Interface(s): one or multiple trusted interfaces (e.g. internal networks). If the device has multiple trusted interface, enter a comma-separated list.

  • Untrusted Interface(s): one or multiple untrusted interfaces (e.g. external networks). If the device has multiple untrusted interface, enter a comma-separated list. If you have enabled FortiOS’s SD-WAN feature, the name of the virtual interface is virtual-wan-link.

  • Send logs to ThreatSTOP: If enabled, the TSCM will send logs received from the device to the ThreatSTOP reporting system. This is the recommended setting. When disabled, logs for this device will not be available for reporting in the Portal.

  • Device username: the username for a Fortigate account available via SSH

  • Device password: the SSH password for the Fortigate account

Advanced Settings

The TSCM supports the following advanced settings, which cover uncommon Fortigate configurations or network environments. To enable the advanced settings, add the –advanced option to the tsadmin command line

$ tsadmin add --type fortigate --advanced [device nickname]

• DNS Port: The TSCM uses TCP Port 53 (outbound connections) to retrieve policy data. If this port is blocked or filtered (for example, networks using a DNS Application Layer Gateway), use this setting to switch to TCP Port 5353.

• Additional syslog source IP(s): Typically, logs will sent over syslog by the device itself. If logs are sent by another IP address (for example, after being processed by a SIEM, or in High-Availability configurations), that IP address should be configured in this field.

• Log rotate size: the TSCM will upload logs after 15 minutes and when the log file size is reached. For systems under very heavy network traffic with many blocked connections, lowering this value will cause logs to be uploaded more often.

• Enable policy updates: this setting can be used to temporarily disabled policy updates by the TSCM. This is not recommended but can be used if device configuration changes needed to be suspended.

• Syslog Forward: if you want the TSCM to forward the logs to another systems (in addition to uploading them to ThreatSTOP’s cloud for reporting), enter yes. The CLI will prompt for the IP address, Port and protocol (UDP or TCP) that logs should be forwarded to.

• Log Upload Proxy: If your environment requires using a proxy to reach HTTPs URLs, you can specify the address of a proxy. The proxy must support HTTPs using the CONNECT protocol. The proxy address must be http://address:port, where address is either an IP address or a fully-qualified domain name. HTTPs proxies are not supported. If you provide a proxy URL, the TSCM configuration will also prompt you for an optional user and password. Provide them if the proxy requires authentication.

• Rules Visibility: By default, the Fortinet Web GUI will display the details of the policy (list of IP addresses). To hide them, set this setting to No. This doesn’t change the policy being applied. The setting takes effects the next time the policy is updated. Also see Fortinet’s documentation.

• Log Upload Proxy: If your environment requires using a proxy to reach HTTPs URLs, you can specify the address of a proxy. The proxy must support HTTPs using the CONNECT protocol. The proxy address must be http://address:port, where address is either an IP address or a fully-qualified domain name. HTTPs proxies are not supported. If you provide a proxy URL, the TSCM configuration will also prompt you for an optional user and password. Provide them if the proxy requires authentication.

At this time, the TSCM has succesfully configured the device entry.

You can view the list of devices linked on the TSCM image:

$ tsadmin list
| Device name | Type | Device ID     | Management IP | Log upload ID | Log  | Log uploads |
| tstest      | fortigate  | tdid_abcd1234 | 172.21.50.3   | tdid_abcd1234 | 100k | enabled     |

• From this point on, the TSCM will retrieve policy data (IP subnets) and configure them on the Fortigate, every hour.
• To force the initial update and proceed with testing, run the following command

  $ tsadmin update <device name>
  

This will create (if not already created) and populate the two Object Groups defined in the device settings.

To verify that the Object Group have been created, open an SSH session to the device and execute the following commands:

• If you have a VDOM configured:

  config vdom
  edit <vdom name>
  

• followed by

  config firewall policy
  show
  end
  

• and

  config firewall address
  edit "TSb1"
  show
  end
  

  You should find the ThreatSTOP ACL configuration for the interfaces and the IP addresses currently contained in the first block group.

Step 4: Fortigate configuration

With the ACLs groups populated, the last step is to configure the Fortigate. Open the web console and perform the following actions:

• Browse to Policy Objects > Policy > IPv4

• Highlight the rules, right-clik and select Enable

You can test that the policy is correctly applied by pinging bad.threatstop.com (64.87.3.133) through the device, a test IP included in all policies. The ping command should fail.

Logging and Reporting

If log upload is enabled, the TSCM will now upload logs every 15 minutes, as long as there were connections blocked by the policy since the last upload. The logs can be analyzed in the IP Defense Reports 15 minutes after they’ve been uploaded.

To check that the log upload feature is able to reach the server:

• After applying the ACLs, generate log entries by trying to reach our test address through the device. The command should fail to connect.

  curl http://bad.threatstop.com
  

• Run the following command on the TSCM to rotate and upload the log file

  $ tsadmin logs
  

If the command doesn’t find a log file to upload, it will exit immediately:

threatstop@tsclient:~$ tsadmin logs
[INFO ] : Starting log upload client
[INFO ] : Log upload client exited

• Log files are stored in /var/log/threatstop/devices/<device name>/syslog.
• If no log file is present, check the syslog forwarding configuration on the device and check that the ACL is applied.

Successful upload:

threatstop@tsclient:~$ tsadmin logs
[INFO ] : Starting log upload client
[INFO ] : [Uploader] Loading device configuration
[INFO ] : Processing logs for device [devicename]
[INFO ] : Starting ThreatSTOP logupload operation v2.00 at 24/05/2018 19:34:05
[INFO ] : Verifying log file [/var/log/threatstop/devices/forti135/syslog.1] stats
[INFO ] : Processing [/var/log/threatstop/devices/devicename/syslog.1] log file
[INFO ] : Start sending data
[INFO ] : Preparing connection data
[INFO ] : Connecting to https://logs.threatstop.com:443/logupload.pl
[INFO ] : Upload was successful [200 OK]
[INFO ] : Completed processing for device [forti135]
[INFO ] : Finish ThreatSTOP logupload operation at 24/05/2018 19:34:10 after 00:00:05
[INFO ] : Log upload client exited

If the command attempts to upload a log but fails, check the connectivity of the TSCM to ThreatSTOP’s log service, described in the connectivity section of this document.

Additional considerations

Support for multiple devices

High-Availability clusters

If your Fortigate is part of a High-Availability cluster, the configuration will be sync’ed across devices by FortiOS. Logs are routed through the primary node and it is not needed to configure additional Syslog IP addresses.

Other operations

Current configuration

To view the current settings on the TSCM, run

$ tsadmin show <device name>

Configuration changes

• After the initial configuration is completed, settings can be edited on the Admin Portal and will be reflected on the device within 5 minutes, including Policy configuration changes.

• If you change the credentials on the Fortigate device, you will need to run tsadmin configure <device name> and update them on TSCM to allow the virtual machine to continue to connect.

The configure command also supports the –advanced option enable the advanced settings described in the configuration section of this document.

Software updates

To update the TIP and retrieve new versions of the ThreatSTOP software, login as threatstop and run the following command:

$ sudo apt-get update && sudo apt-get -y dist-upgrade

Uninstall steps

• To disable the integration, the first step is to disable the rules via the FortiOS web interface, under Policy Objects > Policy > IPv4.

• Next, delete the device entry on the TSCM. This will stop any policy updates or changes to the Fortigate device.

  $ tsadmin remove <devicename>
  

• The last step is to delete the device entry on the Portal, using the Device List page. This step will caused the log data from the device to be unavailable in the Reporting interface of the Portal. If needed, you can recreate a new device entry for the same device, with the same or different settings. Note that the new entry will have a different Device ID for linking the TSCM.

Additional information

Troubleshooting

• Failure to retrieve policy: tsadmin add fails with this error: “block or allow list [name] could not be fetched from ThreatSTOP DNS servers.” There are two common causes:
  • A network connectivity problem using DNS over TCP (Outbound connection to ts-dns.threatstop.com on Port 53).
  • the policy is not available yet. It typically takes 15 minutes for new devices and new policies to be activated in the Policy Service.

• If the network connectivity is ok, and 15 minutes have elapsed since the device entry was created, please contact ThreatSTOP Support at support@threatstop.com.

• If tsadmin add fails to connect to the device, check the credentials (username, password and enable password). If they are correct, check if the ssh password prompt has been changed from Cisco’s defaults (Password:).

Version history

• TSCM

• Fortigate Module

Related documents

• TSCM reference
• Creating custom policies
• Command-line based version of TSCM for Fortigate