Overview

The PCAP integration is a pseudo-device that allows loading packet capture files into the ThreatSTOP reporting system. Unlike regular devices, this feature doesn’t provide a mechanism to apply a policy to the device - only the log analysis and reporting.

With this feature, you can take a snapshot of your network traffic and generate reports to assess if connections are being made from or to IP addresses listed in ThreatSTOP Threat Intelligence database.

The PCAP data must be uploaded in ASCII format; binary PCAP files will not be processed.

Detection of traffic direction

The PCAP format doesn’t provide an indication of whether the packets were sent from your network to the Internet (outbound), or from the Internet to your network (inbound). By default, the ThreatSTOP Log parser will mark the packets as sent inbound. However, if the packet capture is done inside the network and detects RFC 1918 IPs (private IP addresses), the parser will mark packets originating from private IPs to public IPs as outbound.

Setup

  • Create a device entry in the ThreatSTOP Admin Portal with the following type:
    • Type: IP Device
    • Manufactuer: Packet Capture
    • Model: PCAP
  • Provide the following settings:
    • Nickname: name the entry; this will be used to identify logs originated from this device in reports.
    • IP Type: Select ‘Static’ to identify the device using its public IP address or select ‘Dynamic’ to use a DNS name pointing to the IP address if it’s dynamic
    • IP Address: the static, public IP address of the device
    • Domain name: a DNS Fully-Qualified Domain Name (A Record) that is kept up to date with the dynamic IP of the device
    • Policy: while a policy will not be applied to the device, the parsing of the log file performed upon upload will use the IP address IOCs contained in the policy when looking for IOCs in the log.
    • Note: an optional note about the device.

Uploading logs

Log files

  • The system will detect any PCAP files that contains the timestamp, source IP address, source port, destination IP address and destination port in the standard format. Additional fields will be ignored.

tcpdump command:

$ sudo tcpdump -tt -n
  • -tt provides the timestamps as seconds since Jan 1, 1970 at 00:00:00 UTC
  • -n shows IP addresses and ports in numeric format

Sample output:

1517931955.781702 IP 10.0.2.2.53231 > 192.1.2.3.80: Flags [.], ack 184924, win 65535, length 0
  • The maximum size of the PCAP file (in ASCII) that can be uploaded is 15 MB.

Manual upload

To upload logs for the device created above, follow these steps:

  • Login to the ThreatSTOP Admin Portal
  • Browse to the Logs menu, and then the User Log Submission tab
  • Identify the device by its nickname or IP address in the list of devices
  • Choose a file to upload and click upload

Files are processed within 15 minutes of upload and their data becomes available in the IP Defense reports.

Automated upload (Linux)

You can also automated the upload of logs from a Linux system. You will need root access.

  • Allow tcpdump to execute post-rotate scripts (Ubuntu only) 
    $ sudo apt install apparmor-utils
$ sudo aa-complain /usr/sbin/tcpdump

  • Create a shell script named tsupload.sh in a directory of your choice

    #!/bin/bash

tmpfile=tcpdump-ascii.$$
tcpdump -n -tt -r $1 >> $tmpfile
/usr/bin/curl -F "upfile=@$tmpfile" -F "upfile_size=`/usr/bin/stat -c %s $tmpfile`" -F "md5_client=`/usr/bin/md5sum $tmpfile|/usr/bin/cut -d' ' -f 1`" https://logs.threatstop.com/cgi-bin/logupload.pl
rm $tmpfile
  • Run tcpdump 
    sudo tcpdump -tt -n -G 600 -w 'tspcap.%s' -Z `whoami` -z /path/to/tsupload.sh
  • The command will rotate a new PCAP file every 600 seconds and upload the ASCII version via HTTPs.
  • On a traffic with high volume of data, the ASCII files might exceeed the 15 MB limit. You can rotate files more often or add a filter to the tcpdump command. For example, to filter TCP connection packets only, use this filter: 
    "tcp[tcpflags] & (tcp-syn|tcp-ack) != 0"