This document provides examples for managing user-defined lists using the ThreatSTOP API.

Overview

API requests

  • A ThreatSTOP REST API token with Read/Write permissions on User Lists objects is required
  • Tokens are generated using the ThreatSTOP Admin Portal User interface (API Keys page)
  • The API token is tied to an account, and can manage the lists from the account it is associated with. To manage lists in multiple accounts, a token from each account must be used.
  • API Endpoint: https://rest.threatstop.com/v4.0/
  • Authentication is performed using Bearer tokens, provided as an HTTP header 
    Authorization: Bearer <Token>
  • Authentication error codes
    • The API returns HTTP/401 if the token is missing
    • The API returns HTTP/403 if the token is invalid or doesn’t have access to the requested resource
  • Examples are provided using curl (https://curl.haxx.se/) and jq (https://stedolan.github.io/jq/manual/v1.4/) on a Unix system

Object types

  • There are two types of user defined lists - one type for IP addresses and one for Domains. They are managed independently.
  • Domain lists can only be used in DNS Defense (RPZ) policies, while IP lists can be used in both IP Defense and DNS Defense policies.
  • The typical workflow consists of:
    1. Creating the user list, a one time operation, and retrieving its object ID
    2. Assigning it to a policy (not covered in this example, can be done via API or Portal)
    3. Adding/removing entries from the list. The changes can be incremental (adding/removing entries) or a full replacement of the content.
  • User-defined Lists are restricted to a maximum of 32,000 records each. Multiple user-defined lists can be attached to a policy.

Data format

  • Object IDs follow the UUID Format
  • List names within an account must be unique
  • List Names must be an alphanumerical string between 1 and 8 characters long
  • IP addresses include a netmask (a.b.c.d/n)
  • Expiration dates are written as DD/MM/YYYY
  • Responses contain a HATEOAS subdocument as _links

IP Lists

  • Endpoint: https://rest.threatstop.com/v4.0/user_ip_lists/
  • To protect against mistakes, IP User-Defined lists can’t include a private (RFC1918) IP or subnets, and subnets larger than /12 cannot be added.

Creating a new IP list

  • Request body 
    cat > create_ip_udl.json << EOF
{
  "addresses": [
      {
          "comments": "created via API",
          "value": "1.2.3.4"
      },
      {
          "comments": "will expire soon",
          "expires": "03/29/2018",
          "value": "4.3.2.1/32"
      }
  ],
  "list_name": "api1test",
  "list_type": "block",
  "shared": false
}
EOF
    • description is an optional field (string up to 1024 characters)
    • The expires field is an optional expiration date for the entries in the list.
    • Each record can override the expiration date
    • “shared” is not documented here (sharing of lists across multiple accounts). Parameter is optional, or must be set to False.
  • Execute the request 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_ip_lists' -X POST --header "Content-Type: application/json" --data @create_ip_udl.json
  • Response
    • HTTP/201 (CREATED) on success.
    • HTTP/400 if the request could not be processed
    • HTTP/500 for REST server errors
    • The Object ID of the newly created user list is accessible as ._data[0].object_id.
  • Sample 
    {
  "_data": [
      {
          "_links": {
              "self": {
                  "href": "https://rest.threatstop.com/v4.0/user_ip_lists/51e8c042-6f0a-4186-8f7c-a8441e2909f3"
              }
          },
          "addresses": [
              {
                  "address_type": "ip",
                  "comments": "created via API",
                  "expires": null,
                  "value": "1.2.3.4"
              },
              {
                  "address_type": "netmask",
                  "comments": "will expire soon",
                  "expires": "2018-03-29",
                  "value": "4.3.2.1/32"
              }
          ],
          "description": "",
          "list_name": "api1test",
          "list_type": "block",
          "object_id": "51e8c042-6f0a-4186-8f7c-a8441e2909f3",
          "shared": false
      }
  ],
  "_links": {
      "self": {
          "href": "https://rest.threatstop.com/v4.0/user_ip_lists"
      }
  }
}
  • With jq: 
    cat response.json | jq -r "._data[0].object_id"
51e8c042-6f0a-4186-8f7c-a8441e2909f3
  • It is possible to create records during the list creation by adding addresses in the same format as described in the Add Records below. See the Python 3 example at the end of this document.

Retrieving the list of IP user defined lists

  • You can retrieve the lists of User-defined lists with a GET request.
  • The response will be an array with the user-defined lists available in the account (same format as returned by the creation request).
  • You can retrieve the Object ID as ._data[entry_number].object_id 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_ip_lists' -X GET
  • Response codes
    • HTTP/200 on success
  • Sample response 
    {
  "_data": [
      {
          "_links": {
              "self": {
                  "href": "https://rest.threatstop.com/v4.0/user_ip_lists/368ecff4-a524-4b24-a261-77ed271fdf1c"
              }
          },
          "_meta": {
              "addresses": {
                  "address_count": 413,
                  "record_count": 413
              }
          },
          "description": "",
          "list_name": "bruteforce",
          "list_type": "block",
          "object_id": "368ecff4-a524-4b24-a261-77ed271fdf1c",
          "shared": false
      }
  ],
  "_meta": {
      "count": 1
  }
}
  • The meta data section contains the number of user lists (as ._meta.count)
  • Each lists object also includes a metadata section with:
    • the number of records in the list (._data[0]._meta.addresses.record_count)
    • the number of IP addresses in the list (._data[0]._meta.addresses.address_count), computed using the size of the subnets it contains

Managing records

Endpoint

  • https://rest.threatstop.com/v4.0/user_ip_lists/{ Object ID }

Retrieving records

  • Records are retrieved with a GET request 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_ip_lists/368ecff4-a524-4b24-a261-77ed271fdf1c' -X GET

Adding a record

  • Sample request 
    cat > add_ip.json << EOF
{
  "addresses": [
      {
          "value": "1.2.3.4",
          "action": "add"
      }
  ]
}
EOF
  • Execute the request 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_ip_lists/8c4350db-4439-4458-9687-af7dc2c877e6' -X PATCH --header "Content-Type: application/json" --data @add_ip.json
  • Response codes
    • 200 on success
    • 400 if the request could not be processed

Deleting a record

  • Sample request 
    cat > del_ip.json << EOF
{
  "addresses": [
      {
          "value": "1.2.3.4",
          "action": "remove"
      }
  ]
}
EOF
  • Execute the request 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_ip_lists/8c4350db-4439-4458-9687-af7dc2c877e6' -X PATCH --header "Content-Type: application/json" --data @del_ip.json
  • Response codes
    • 200 on success
    • 400 if the request could not be processed

Updating a record

  • Sample request 
    cat > update_ip.json << EOF
{
  "addresses": [
      {
          "value": "1.2.3.4",
          "comments": "new comment"
      }
  ]
}
EOF
  • Execute the request 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_ip_lists/8c4350db-4439-4458-9687-af7dc2c877e6' -X PATCH --header "Content-Type: application/json" --data @update_ip.json
  • Response codes
    • 200 on success
    • 400 if the request could not be processed

Domain Lists

  • Endpoint: https://rest.threatstop.com/v4.0/user_domain_lists/
  • Requests and responses use the same format as IP list

Creating a new domain list

  • Sample request 
    cat > create_domain_udl.json << EOF
{
  "addresses": [
      {
          "comments": "created via API",
          "value": "example.com"
      },
      {
          "comments": "will expire soon",
          "expires": "03/29/2018",
          "value": "sub.example.com"
      }
  ],
  "list_name": "api2test",
  "shared": false
}
EOF
    • The expiration date is an optional expiration date for the entries in the list.
    • Each record can override the expiration date
    • “shared” is not documented here (sharing of lists across multiple accounts). Set to false
  • Execute the request 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_domain_lists' -X POST --header "Content-Type: application/json" --data @create_domain_udl.json
  • Response
    • HTTP/201 (CREATED) on success.
    • HTTP/400 if the request could not be processed
    • HTTP/500 for REST server errors
    • The Object ID of the newly created user list is accessible as ._data[0].object_id.

  • The response object has the same format as the IP list object

  • Retrieving the object ID of the new list with jq: 
    cat response.json | jq -r "._data[0].object_id"
51e8c042-6f0a-4186-8f7c-a8441e2909f3
  • It is possible to create records during the list creation by adding addresses in the same format as described in the Add Records below. See the Python 3 example at the end of this document.

Managing records

Endpoint

  • https://rest.threatstop.com/v4.0/user_domain_lists/{ Object ID }

Retrieving records

  • Records are retrieved with a GET request 
    curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_domain_lists/cd358077-5ecb-40a2-aec3-82b7ad880d8d' -X GET

Adding a record

curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_domain_lists/8612ab35-118c-4afd-9891-cf32b019c219' -X PATCH --header "Content-Type: application/json" --data '{"addresses": [{"value": "example.com", "action": "add"}]}'

Deleting a record

curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_domain_lists/8612ab35-118c-4afd-9891-cf32b019c219' -X PATCH --header "Content-Type: application/json" --data '{"addresses": [{"value": "example.com", "action": "remove"}]}'

Updating a record

curl --silent --header 'Authorization: Bearer <API Token>' 'https://rest.threatstop.com/v4.0/user_domain_lists/8612ab35-118c-4afd-9891-cf32b019c219' -X PATCH --header "Content-Type: application/json" --data '{"addresses": [{"value": "example.com", "comments": "new comment"}]}'

Python 3 Example

Pre-requisites

  • Create a virtual environment 
    virtualenv -p python3 env
  • Install the requests package 
    pip install requests

Sample code

#!/usr/bin/env python3
import sys
import requests

api_url = 'https://rest.threatstop.com/'
api_version = 'v4.0'
# put your API token here
api_key = 'YOUR_API_TOKEN'

ip_list_name = 'apitestip'
domain_list_name = 'apitestdom'

# Create a new user IP list with 2 records
response = requests.post(
    url='{api_url}/{version}/{endpoint}'.format(
        api_url=api_url,
        version=api_version,
        endpoint='user_ip_lists'
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "list_name": ip_list_name,
        "list_type": "block",
        "shared": False,
        "addresses": [
            {
                "value": "1.2.3.4",
                "comments": "created via API"
            },
            {
                "value": "4.3.2.1/32",
                "comments": "will expire soon",
                "expires": "03/29/2018"
            }
        ]
    }
)

# Response handling
if response.status_code == 201:
    data = response.json()['_data'][0]
    print("User IP list {} created; Object ID {}".format(
        data['list_name'], data['object_id']
    ))
else:
    print("Unable to create user list: {}".format(
        response.json()
    ))
    sys.exit(101)

# Create user domain list with 2 records
response = requests.post(
    url='{api_url}/{version}/{endpoint}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_domain_lists'
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "list_name": domain_list_name,
        "shared": False,
        "addresses": [
            {
                "value": "example.com", 
                "comments": "created via API"
            },
            {
                "value": "sub.example.com", 
                "comments": "will expire soon", 
                "expires": "03/29/2018"
            }
        ]
    }
)

# Response handling
if response.status_code == 201:
    data = response.json()['_data'][0]
    print("User domain list {} created; Object ID {}".format(
        data['list_name'], data['object_id']
    ))
else:
    print("Unable to create user list: {}".format(
        response.json()
    ))
    sys.exit(101)

# Find Object ID of a newly-created IP list. 
# It can also be found in the POST response
response = requests.get(
    url='{api_url}/{version}/{endpoint}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_ip_lists'
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)}
)

if response.status_code == 200:
    for record in response.json()['_data']:
        if record['list_name'] == ip_list_name:
            ip_list_object_id = record['object_id']
            print('Found user ip list {} with name {}'.format(
                record['object_id'], record['list_name']
            ))
            break
    else:
        print('User list {} not found.'.format(ip_list_name))
        sys.exit(102)
else:
    print('Unable to get list of the user lists: {}'.format(
        response.json()
    ))
    sys.exit(102)

# Find Object ID of a newly-created domain list. 
# It can also be found in POST response
response = requests.get(
    url='{api_url}/{version}/{endpoint}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_domain_lists'
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)}
)

if response.status_code == 200:
    for record in response.json()['_data']:
        if record['list_name'] == domain_list_name:
            domain_list_object_id = record['object_id']
            print('Found user domain list {} with name {}'.format(
                record['object_id'], record['list_name']
            ))
            break
    else:
        print('User list {} not found.'.format(domain_list_name))
        sys.exit(102)
else:
    print('Unable to get list of the user lists: {}'.format(response.json()))
    sys.exit(102)

# Delete one IP record
response = requests.patch(
    url='{api_url}/{version}/{endpoint}/{object_id}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_ip_lists',
        object_id=ip_list_object_id
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "addresses": [
            {"value": "1.2.3.4", "action": "remove"},
        ]
    }
)
if response.status_code == 200:
    print('User IP list updated: {}'.format(
        response.json()['_meta']['addresses']
    ))
else:
    print('Unable to patch user IP list: '.format(response.json()))
    sys.exit(103)

# Add one IP record
response = requests.patch(
    url='{api_url}/{version}/{endpoint}/{object_id}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_ip_lists', 
        object_id=ip_list_object_id
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "addresses": [
            {"value": "1.2.3.4", "action": "add"},
        ]
    }
)
if response.status_code == 200:
    print('User IP list updated: {}'.format(
        response.json()['_meta']['addresses']
    ))
else:
    print('Unable to patch user IP list: '.format(response.json()))
    sys.exit(103)
# Add IP record
response = requests.patch(
    url='{api_url}/{version}/{endpoint}/{object_id}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_ip_lists', 
        object_id=ip_list_object_id
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "addresses": [
            {"value": "1.2.3.4", "comments": "new comment"},
        ]
    }
)
if response.status_code == 200:
    print('User IP list updated: {}'.format(
        response.json()['_meta']['addresses']
    ))
else:
    print('Unable to patch user IP list: '.format(response.json()))
    sys.exit(103)
# Delete domain record
response = requests.patch(
    url='{api_url}/{version}/{endpoint}/{object_id}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_domain_lists', 
        object_id=domain_list_object_id
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "addresses": [
            {"value": "example.com", "action": "remove"},
        ]
    }
)
if response.status_code == 200:
    print('User IP list updated: {}'.format(
        response.json()['_meta']['addresses']
    ))
else:
    print('Unable to patch user IP list: '.format(response.json()))
    sys.exit(103)
# Add domain record
response = requests.patch(
    url='{api_url}/{version}/{endpoint}/{object_id}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_domain_lists', 
        object_id=domain_list_object_id
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "addresses": [
            {"value": "example.com", "action": "add"},
        ]
    }
)
if response.status_code == 200:
    print('User IP list updated: {}'.format(
        response.json()['_meta']['addresses']
    ))
else:
    print('Unable to patch user IP list: '.format(response.json()))
    sys.exit(103)
# Update domain record
response = requests.patch(
    url='{api_url}/{version}/{endpoint}/{object_id}'.format(
        api_url=api_url, 
        version=api_version, 
        endpoint='user_domain_lists', 
        object_id=domain_list_object_id
    ),
    headers={'Authorization': 'Bearer {}'.format(api_key)},
    json={
        "addresses": [
            {"value": "example.com", "comments": "new comment"},
        ]
    }
)
if response.status_code == 200:
    print('User IP list updated: {}'.format(response.json()['_meta']['addresses']))
else:
    print('Unable to patch user IP list: '.format(response.json()))
    sys.exit(103)

Extended error codes

If the API returns an HTTP error (typically, status code 400), it will include a JSON object with details about the error.

  • Error format 
    {
  "additional_info": {
      "detail": "<error details>",
      "error_code": <extended_error_code>
  },
  "error_description": "<message>",
  "status_code": 400
}
Code Meaning
10301 POST request with an Object ID
11000 No auth token provided
11001 Invalid auth token
11002 Expired Auth token
11400 Bad parameters in request
19000 User List name already in use
19011 Exceed maximum number of addresses
19013 Domains exceeded the maximum length (200 char)
19012 Request included forbidden network entry
19050 Invalid address type in request