Juniper SRX (Security Zones) | ThreatSTOP Dochub

This document describes the integration process of the ThreatSTOP IP Defense with Juniper SRX Devices (Address Book API)

Overview

We have written some scripts to set up the SRX with the correct firewall rules, to get your block lists, use the results to update the rules and to upload your firewall logs to us.

You can download the scripts here but you will probably find it easier to cut and paste the line in the instructions below.

If this is a new device, please allow up to 15 minutes for our systems to be updated.

Prerequisites

The scripts do not work when using the global address-book in JunOS. If you are using the address books created in security address-book, they will need to be moved to the security zone before running the scripts.

Recommended Versions

ThreatSTOP is compatible with most versions of JunOS.

Minimum	11.4R7.5
Recommended	12.1X46-D35.1

You should connect to the SRX and log in to it as root. Using that login session you should confirm that the SRX can also access and download files from the Internet. In particular you should check that your SRX can:

ping ftp.threatstop.com

It is recommended that you save the configuration prior to applying the ThreatSTOP changes, which can be done by entering from the console:

cli
configure
save prethreatstop
exit
exit

The Files

tsinstall.sh: installation script.
tsinstall.xsl: supporting XSl script called from tsinstall.sh.
tsuninstall.sh: uninstallation script.
tsuninstall.xsl: supporting XSl script called from tsuninstall.sh.
tsgetblocklist.sh: sctript that runs periodically to download new blocklist.
ipget_head.xml: supporting XSl script called from tsgetblocklist.sh.
ipget_tail.xsl: supporting XSl script called from tsgetblocklist.sh.
ipget-head_trust.xsl: supporting XSl script called from tsgetblocklist.sh.
ipget_head.xsl: supporting XSl script called from tsgetblocklist.sh.
ipget-head_untrust.xsl: supporting XSl script called from tsgetblocklist.sh.
ipget_tail.xml: supporting XSl script called from tsgetblocklist.sh.
threatstop.conf: configuration file.
tspolicycreator.sh: ThreatSTOP policy creation script.

Install

Copy and paste the following line into the SRX ssh session:

cd ~ && fetch -p ftp://ftp.threatstop.com/pub/ts-juniper.tar.gz && tar zxvf ts-juniper.tar.gz && cd ts-juniper && echo "BLOCKLIST=<block list name>.<ThreatSTOP account ID>.threatstop.local" >> threatstop.conf && echo "ALLOWLIST=<allow list name>.<ThreatSTOP account ID>.threatstop.local" >> threatstop.conf && echo "DEVICE_IP="<Device IP>"" >> threatstop.conf && /bin/sh tsinstall.sh

When the install script is run, it will ask for the names of the trusted and untrusted zones. If you have renamed the default zones, enter the new names at the prompt. If you have not renamed the default zones, you can hit ENTER when the script asks for the zone names.

You have the following zones
Security zone: trust
Security zone: untrust
If your UNTRUST zone is not 'untrust',
    please enter the new name here: 
Using untrust zone: untrust
ThreatSTOP block address sets will be created in the 'untrust' zone
If your primary TRUST zone is not 'trust',
    please enter the new name here: 
Using trust zone: trust
ThreatSTOP address sets will be created in the 'trust' zone
root@srx> configure
Entering configuration mode
[edit]
root@srx# set system scripts commit file tsinstall.xsl
[edit]
root@srx# commit check
configuration check succeeds
[edit]
root@srx# commit
commit complete
[edit]
root@srx#
[edit]
root@srx#
root@srx> configure
Entering configuration mode
[edit]
root@srx# set system scripts commit file tsipget-untrust.xsl
[edit]
root@srx# commit
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-1
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-2
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-3
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-4
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-5
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-6
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-7
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-8
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-9
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-10
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-11
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-12
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-13
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-14
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-15
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-16
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-17
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-18
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-19
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-20
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-21
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-22
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-23
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-24
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-25
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-26
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-27
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-28
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-29
warning: New address set is created containing up to 1000 block addresses
ThreatSTOP-block-30
commit complete
[edit]
root@srx#
[edit]
root@srx#
root@srx>configure 
Entering configuration mode
[edit]
root@srx# set system scripts commit file tsipget-trust.xsl
[edit]
root@srx# commit
warning: New address set is created containing up to 1000 allow addresses
ThreatSTOP-allow-1
commit complete
[edit]
root@srx#
[edit]
root@srx#
ThreatSTOP address books have been created on your device.
Cron job to update the block lists is configured to run every 2 hours.
To fully enable ThreatSTOP you will need to run the
'tspolicycreator.sh' script to create the policies.
root@srx%

To take into account the growing and shrinking of the block list, we automatically create 30 address book sets in the configuration. In address sets that would normally be empty, we add a network that would normally not be seen, 169.254.0.0/16.

In order to have the block lists updated at regular intervals, it creates an event script that runs every 2 hours and which updates the address book entries. It also modifies the syslog configuration so as to:

Log firewall block messages to ..log.
Upload ..log to the ThreatSTOP log server every eight hours or when the log is greater than 100KB

Adding Policies

Since every firewall configuration is different, ThreatSTOP does not automatically generate the policies you need. A utility script called tspolicycreator.sh is available in the ts-juniper directory that will generate appropriate lines in the configuration. This script is not run automatically because its output should be checked before it is applied.

/sbin/sh tspolicycreator.sh

Here is a sample of the tspolicycreator.sh script:

If your UNTRUST zone is not 'untrust', please enter the new name here:
Using the following UNTRUST zone: untrust

If your TRUST zone is not just 'trust', please enter the new name (or names) here:
using the following TRUST zone(s): trust

configure
set security policies from-zone trust to-zone untrust policy ThreatSTOP-allow-A match source-address any
set security policies from-zone trust to-zone untrust policy ThreatSTOP-allow-A match destination-address ThreatSTOP-allow-A
set security policies from-zone trust to-zone untrust policy ThreatSTOP-allow-A match application any
set security policies from-zone trust to-zone untrust policy ThreatSTOP-allow-A then permit

set security policies from-zone untrust to-zone trust policy ThreatSTOP-allow-A match destination-address any
set security policies from-zone untrust to-zone trust policy ThreatSTOP-allow-A match source-address ThreatSTOP-allow-A
set security policies from-zone untrust to-zone trust policy ThreatSTOP-allow-A match application any
set security policies from-zone untrust to-zone trust policy ThreatSTOP-allow-A then permit

insert security policies from-zone trust to-zone untrust policy ThreatSTOP-allow-A before policy trust-to-untrust
insert security policies from-zone untrust to-zone trust policy ThreatSTOP-allow-A before policy web-server

set security policies from-zone trust to-zone untrust policy ThreatSTOP-block-X match source-address any
set security policies from-zone trust to-zone untrust policy ThreatSTOP-block-X match destination-address ThreatSTOP-block-X
set security policies from-zone trust to-zone untrust policy ThreatSTOP-block-X match application any
set security policies from-zone trust to-zone untrust policy ThreatSTOP-block-X then deny
set security policies from-zone trust to-zone untrust policy ThreatSTOP-block-X then log session-init
set security policies from-zone trust to-zone untrust policy ThreatSTOP-block-X then count

set security policies from-zone untrust to-zone trust policy ThreatSTOP-block-X match destination-address any
set security policies from-zone untrust to-zone trust policy ThreatSTOP-block-X match source-address ThreatSTOP-block-X
set security policies from-zone untrust to-zone trust policy ThreatSTOP-block-X match application any
set security policies from-zone untrust to-zone trust policy ThreatSTOP-block-X then deny
set security policies from-zone untrust to-zone trust policy ThreatSTOP-block-X then log session-init
set security policies from-zone untrust to-zone trust policy ThreatSTOP-block-X then count

insert security policies from-zone trust to-zone untrust policy ThreatSTOP-block-X before policy trust-to-untrust
insert security policies from-zone untrust to-zone trust policy ThreatSTOP-block-X before policy web-server

commit check
commit
exit

If the policies above are correct enter YES to commit them to the config.

If you are happy with the policies that are created, you can type YES to have the script apply the changes. If you want to review the policies that will be created in more detail before applying them, you can view the list of commands that will be run in the following files:

/tmp/alopol: policies for the allow list
/tmp/blockpol: policies for the block list

To view the policies that will be applied, run:

less /tmp/alopolless /tmp/blockpol

If you need to edit the files:

vi /tmp/alopol
vi /tmp/blockpol
If you are happy with them, you can apply the policies with the following commands:
cat /tmp/alopol | /usr/sbin/cli
cat /tmp/blockpol | /usr/sbin/cli

Cluster Setup

In a cluster environment, the script needs to be installed on all the cluster nodes. The scripts will only run on the primary node. Due to how SRX clusters work, the secondary nodes may not be able to access the internet, and it will be easier to copy the script from the primary node. To copy the scripts to the secondary nodes, access the node with SSH and run:

scp root@IP_ADDRESS_OF_PRIMARY_NODE:~/ts-juniper.tar.gz

Replace “IP_ADDRESS_OF_PRIMARY_NODE” with the IP address of the primary node. After the script is copied, extract it and run the tsinstall.sh script:

tar zxvf ts-juniper.tar.gz && cd ts-juniper && echo "BLOCKLIST=<block list name>.<ThreatSTOP account ID>.threatstop.local" >> threatstop.conf && echo "ALLOWLIST=<allow list name>.<ThreatSTOP account ID>.threatstop.local" >> threatstop.conf && echo "DEVICE_IP="<Device IP>"" >> threatstop.conf && /bin/sh tsinstall.sh

When the tsinstall.sh script is run, it will automatically detect that it is not the primary node and will only create the cronjob that will run the tsgetblocklist.sh script. This script will also detect that it is not the primary node and will not run. It will only run if the cluster failed over and the node becomes primary.

Is this being installed on a cluster?(y/n) [n]: y
This is the secondary node
Cron job to update the block lists is configured to run every 2 hours.

Troubleshooting

The following four problems are frequently seen by our support staff, if you are confused or if these steps do not help then please contact ThreatSTOP support.

ThreatSTOP rules do not appear to download anything The most likely reason is that you have not correctly entered the firewall’s IP address in the ThreatSTOP device definition page. You can verify whether the address the firewall uses is in our database by visiting the follwing URL in a browser that is NATted through the firewall:

http://logs.threatstop.com/cgi-bin/validip.pl

You should see a simple result stating the device’s IP address and whether it is in the database or not. Database updates are not instantaneous but take place every 20 minutes; so, if you have recently added/modified the firewall IP details, you may wish to wait about half an hour before checking this. If there is no response at all then verify by using the ping command that the firewall can reach threatstop.com and, if not, that it can reach other places such as google.com. If you have no connectivity to ThreatSTOP but do have connectivity elsewhere please contact ThreatSTOP support for further information about the status of the ThreatSTOP infrastructure.

If the address is valid but there are still problems you should manually run the blocklist retrieval procedure and then examine the output file. To download the blocklists do the following at the command prompt (SSH or console):

/bin/sh ts-juniper/tsgetblocklist.shless /tmp/tsblock.addr

The file should contain a list of ip addresses with subnet masks. If the file does not exist and/or an error was printed when you ran the tsgetblocklist.sh command then check that you have connectivity to ThreatSTOP’s DNS servers (listed in ts-juniper/threatstop.conf) and that you have in fact configured some blocklists for this firewall. The list should be the same as the list of addresses in the ‘untrust’ zone address book, you can check this as follows:

cli
show configuration | grep "address ThreatSTOP-block"
exit

If the file exists and is not the same as the address lists then please contact ThreatSTOP support to help debug the issue.

Not all traffic is being blocked / everything is being blocked

The most likely reason is that you have not correctly set the policies using the address sets (see Adding Policies above). If you need help with verifying the policy names the contact ThreatSTOP support.

ThreatSTOP blocks access to places it shouldn’t

Although ThreatSTOP tries very hard to ensure that we have zero false positives in our standard lists, we do occasionally miss something. If you are a community user or are using ThreatSTOP in standard mode on your firewall then please report the offending domain and ip address to ThreatSTOP support. Please check in the log file (/var/log/ ..log) that the IP address is indeed being blocked by the ThreatSTOP rules before contacting us. You can also add the IP address to a custom allow list. Once you have created the allow list and added it to this device in the "Edit device" page. If you did not use an allow list when you created the device, then the first time one is added you should run the following command on the SRX command prompt to enable it on the device:

echo "ALLOWLIST=<allow list name>.<ThreatSTOP account ID>.threatstop.local" >> ~/ts-juniper/threatstop.conf

Note: Our expert lists are not so well checked for false positives as we believe that it is up to each expert user to make his or her decision about the suitability of certain feeds and some feeds - e.g. the Parasites feed - are known to block ip addresses that are not considered harmful by everyone. Do consider a custom allowlist (see above) as a first step and do please contact ThreatSTOP support to verify that the feeds you have chosen are appropriate to your situation.

Other firewall rules do not appear to work

The most common reason for this is the same as 2) above, namely that the ThreatSTOP policies have been incorrectly set up.

No Logs Uploaded

In many cases logging issues are related to issue 1) above. If running the “validip” test described above is successful then you should verify that “log enable” is set appropriately in the current configuration (all ThreatSTOP firewall drop policies should include a log command in them).

You should also confirm that the file /var/log/ ..log exists. If it does not please contact ThreatSTOP support. finally check that the syslog handling part of the configuration is set to upload the ..log file to ftp://logs.threatstop.com/logs

file <block list name>.<ThreatSTOP account ID>.log{
any any;
match RT_FLOW_SESSION;
archive transfer-interval 480 archive-sites {
"ftp://ftp@logs.threatstop.com/logs"}
}

The logs are transferred to ThreatSTOP using FTP. To allow the SRX to FTP the logs to us, the FTP Application Layer Gateway (ALG) needs to be enabled. By default, the FTP ALG is enabled. To see the status of the FTP ALG, run: show security alg

If it is not listed, then it is disabled. If it is listed, you can enable it by running:

delete security alg ftp
commit

Restore to previous state

If you have run our install script, applied the policy changes and wish to return to the pre-ThreatSTOP configuration then you should perform the following actions.

To restore the configuration you should enter from the console:

cli
configure
load override presthreatstop
save
exit
exit

Note: This will also undo any other changes made to the configuration since ThreatSTOP was installed.

Now to uninstall the ThreatSTOP scripts enter:

/bin/sh ~/ts-juniper/tsuninstall.sh

Optionally you may also wish to remove the ts-juniper directory (rm -r ts-juniper/).

This has now restored all the files changed.

For sanitation you will also want to check the following directories for any ThreatSTOP related files that have been left behind and remove them:

/tmp/*
var/log/<Device IP>.log
var/db/scripts/commit/
var/db/scripts/op/
/tmp/*
var/log/<ip address>.log
var/db/scripts/commit/
var/db/scripts/op/