Table of contents

Welcome to the early version of Toolsaf, a tool-driven security assessment framework. Toolsaf is for security assessment of Internet of Things (IoT) and other systems using different security tools. Security assessment process using Toolsaf has two main phases:

1. Creation of a security statement which describes security-related features of an IoT product or system.
2. Verification of the security statement with security tools, thus confirming the security posture of the product or system.

Some of the applications of this approach could be:

1. Security testing and regression testing of IoT systems.
2. Communicating system security properites in a machine-readable and verifiable format.
3. Documenting results of a system's reverse-engineering.

Toolsaf is an open-source project driven by Test of Things. It was originally created as part of Rauli Kaksonen's doctoral thesis "Transparent and tool-driven security assessment for sustainable IoT cybersecurity" under the name of 'tcsfw'. The thesis introduces Tool-driven security assessment (TDSA) to enhance the cybersecurity of IoT products and systems.

The basic idea of TDSA is to first describe the IoT system or product with the security statement, then use tools to verify that the security statement is accurate, and finally use the statement for different purposes. A security statement provides a machine-readable description of the system's security, which allows automated verification.

At the moment, security statements are created by Toolsaf using a dedicated Python-based Domain-Specific Language (DSL). (JSON-based format is under development). Security statement verification is done by running any of the supported tools and saving their output. Toolsaf uses the tools' output to verify that the security statement is accurate. Some tools perform their own security checks, which can be incorporated into the security statement verdict.

The following shows how to start using Toolsaf. For this, you need to perform two tasks:

1. Install the Toolsaf Python module.
2. Initialize a new or copy an existing security statement.

You need a recent Python interpreter, such as 3.10 or newer. Most Linux distributions should already come with a suitable one, the command name might be python3 to separate from older Python interpreters.

Create virtual environment to install Toolsaf and run it. Then install from PyPI with 'pip'.

python3 -m venv .venv
source .venv/bin/activate   # Activate the virtual environment
pip install toolsaf

Now that Toolsaf is installed, you have to create a directory for your first security statement. The commands below produce the expected project structure for your statement.

mkdir statement-device
cd statement-device
mkdir product
touch product/statement.py  # This is the 'security statement'!

Alternatively, you may copy an existing security statement. In that case you must also create and activate the virtual environment and install the Toolsaf module.

If you want to visualize your security statements you also need to install Graphviz.

In the above, we used venv virtual environments, but as security statements are essentially Python-code, any other environment should work fine.

To recap, a security statement is Python DSL code in its own directory, which uses the Toolsaf Python module. There are two basic approaches to create a security statement:

1. Write the security statement code first, then run tools and verify it.
2. Run tools to collect information first, then write security statement to match the findings.

If you’re the device manufacturer you will likely use the former approach, as you (hopefully) know your product's attack surface and other properties. If you are a security researcher who is reverse-engineering or pen-testing a product, you have to use the latter approach.

The following documents describe security statements creation and verification:

• Creating Security Statements
• Verifying Security Statements

Toolsaf is a command-line tool, and its command-line options are listed here.

Minimal security statement examples are available in the samples directory. They can be executed like this:

python samples/device-backend/statement.py

The command outputs basic information about the security statement.

Security statements for real devices are available in their own repositories:

• Ruuvi Gateway & Tags
• Deltaco Smart Outdoor Plug
• IPC360

Contact Toolsaf team by mail toolsaf at testofthings.com

The project is published with MIT license.

We are happy to accept contributions to Toolsaf, please see the instructions.