«
Apache Parquet is an open-source file format that stores data efficiently in columnar format, provides different encoding types, and supports predicate filtering. With good compression ratios and efficient encoding, VPC flow logs stored in Parquet reduce your Amazon S3 storage costs.
» ― AWS Blog

Let’s take advantage of the fact that AWS VPC Flow Logs can be natively stored in Apache Parquet format to seamlessly visualize our network and understand traffic patterns, identify security issues, audit usage, and diagnose network connectivity.

In AWS VPC console select a VPC, click Create flow log and set the following parameters:

1. Send to an Amazon S3 bucket
2. Custom format : standard attributes
3. Log file format : Parquet

Then you can open in Squey as many Parquet files as you want in one go, as show in the quick video below:

“We have published one year of ssh logins/logouts of a valid administrator; However the account has been compromised using social engineering similar to the MGM attack which led to a ransomware being deployed.” ― Detecteam

So here is the openssh.log_.zip (mirror) dataset and its associated openssh.log_.zip.format parsing file.

It’s looking like typical OpenSSH logs:

Sep 24 08:46:18 bidizidomo sshd[26168]: Accepted password for iworkinacasino from 173.194.42.31 port 63346 ssh2
Sep 24 08:46:18 bidizidomo sshd[26168]: pam_unix(sshd:session): session opened for user iworkinacasino(uid=1169) by (uid=0)
Sep 24 08:46:18 bidizidomo systemd-logind[515]: New session 8767 of user iworkinacasino.
Sep 24 08:46:18 bidizidomo sshd[26168]: pam_env(sshd:session): deprecated reading of user environment enabled
Sep 24 12:51:42 bidizidomo sshd[26971]: Received disconnect from 173.194.42.31 port 23568 disconnected by user
Sep 24 12:51:42 bidizidomo sshd[26971]: Disconnected from user iworkinacasino 173.194.42.31 port 23568
Sep 24 12:51:42 bidizidomo sshd[26971]: pam_unix(sshd:session): session closed for user iworkinacasino
Sep 24 14:32:41 bidizidomo sshd[44186]: Accepted password for iworkinacasino from 173.194.42.109 port 26603 ssh2
Sep 24 14:32:41 bidizidomo sshd[44186]: pam_unix(sshd:session): session opened for user iworkinacasino(uid=1169) by (uid=0)
...

During the data ingest step, the dataset was enriched in several ways:

1. Two columns were added by splitting the datetime column in two, to separate the date and the time.
2. The column country was computed thanks to the Python API by using the IPs for the lookup.
3. Columns not containing any relevant information were let aside (namely user and service) and rows that were not containing an IP address.

Here is the resulting dataset (containing 2230 rows and 7 columns) loaded in Squey:

Having a time axis whose values are wrapped around 24h (00:00:00 at the bottom of the axis to 23:59:59 at the top) is very handy to observe the repartition of the connections during the day.

So except for the 7 wrong passwords that were supplied during the connections and the 7 during the reconnections, the connection pattern is disturbingly consistent, as every single day of the year unfolds as follows:

1. start of the day : first connection from the US between 07:00 and 09:00
2. lunch : deconnection and reconnection from the US between 10:56 and 14:58
3. end of the day : deconnection between 16:12 and 19:01

But beyond this extreme conscistency, a particular network traffic visually stands out completely as the time and location of theses connections are totally contrasting:

Indeed, four SSH connections were issued with success from China on May 23rd between 01:08:12 and 04:32:20 :

1. at 01:18:12 (lasted 1m44s)
2. at 01:24:20 (lasted 52m12s)
3. at 02:33:28 (lasted 43m42s)
4. at 03:34:28 (lasted 57m52s)

Although this dataset is extremely simple, it perfectly illustrates the importance of visually presenting data in an optimal fashion to highlight suspicious behaviors.

Creating the parsing file

For those interested in the way the parsing format file was created, here a quick step-by-step video tutorial.

Here are also the commands to install the required Python modules.

flatpak run --command=/app/bin/squey_sh org.squey.Squey
pip3 install python-geoip-python3 python-geoip-geolite2

As they are used by the following embedded Python script.

 1from geoip import geolite2
 2import numpy as np
 3def country(ip):
 4    return geolite2.lookup(ip).country
 5source = squey.source("openssh.log_.zip")
 6ips_column = source.column("IP")
 7unique_ips = np.unique(ips_column).tolist()
 8unique_countries = np.array(list(map(country, unique_ips)))
 9countries = np.array([unique_countries[unique_ips.index(ip)] for ip in ips_column])
10source.insert_column(countries, "country")

As an example, we will load the complete MACCDC 2012 PCAP dataset composed of 17 files (~17GB) and export HTTP communications between IPs 192.168.203.63 and 192.168.229.101 on port 80.

Loading packet captures

Click on the Pcap... button under the SOURCES section of the start page and click on the Manage profiles button.

First, let choose the packet fields we would like to use to make our filtering. Almost all fields supported by Wireshark are available, but keep in mind that the more fields you choose, the slower the packet captures will be loading and the bigger the space in RAM will also be.

Click on New profile, enter the profile name of your choice and click Ok.

Then click Select and browse one of the PCAP of the dataset.

For this exemple we chose the following fields:

frame.time, eth.dst, eth.src, ip.dst, ip.src, tcp.srcport, tcp.dstport

And we also checked the Protocol field in the Options tab.

Save your profile and load the packets capture using the newly created profile.

The packet captures should now be displayed using parallel coordinates with the fields you selected as columns.

Isolating packets

In order to isolate this communication, we will successively apply 4 stages of filtering:

1. Filtering packets using HTTP protocol
2. Filtering packets to IPs 192.168.229.101 and 192.168.203.63
3. Filtering packets from IPs 192.168.229.101 and 192.168.203.63
4. Filtering packets on destination port 80

1. Right-click on the _ws.col.Protocol column header, select Distinct values, click on HTTP and close the dialog.

2. Right-click on any value of the ip.dst column, select Search for... paste the two IPs addresses located below and click Apply.

192.168.229.101
192.168.203.63

3. Repeat the same filtering operation on the ip.src column.

4. Right-click on the tcp.dstport column header, select Distinct values, click on 80 and close the dialog.

Exporting isolated packets

Now that the communication is properly isolated, time to export it back as a PCAP file.

Click File > Export > Selection... and save the PCAP wherever you want.

The application has just filtered the packets contained in the original PCAP files.
The default option Export complete TCP steams will also export the entire session a packet belongs to.

You can now open the exported PCAP with Wireshark or your favorite tools.

Saving the investigation for faster reloads

If you want to avoid the loading time of the packet captures, saving it as an investigation will let you reload it very quickly (in less than 6 seconds on an Intel® Core™ i7-12700H Processor using ~3.6GB of data on disk).

Click File > Investigation > Save investigation (or Ctrl+S) and select the folder you want to save your investigation to.
Note that the actual data of the investigation will still be located in the application temporary directory.

The investigation will then appear under the INVESTIGATIONS section of the start page as a clickable link.

Deleting the investigation to free up disk space

Tick the investigation checkbox, click Delete and chose Delete investigation.

Disk space will automatically be reclaimed during application next loading.

Loading the dataset

Click on the Local files... button located on the SOURCES section of the start page and browse the compressed dataset.

The file format and column types will be automatically detected, so just click Yes and Save.

Unfortunately the dataset isn’t containing a header with column names but don’t worry, we got them from the dataset description:

ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto port query qclass qclass_name qtype qtype_name rcode rcode_name QR AA TC RD Z answers TTLs rejected

Choose Yes, paste the column names provided above, click Ok, then click Save.

We now have the dataset loaded with the proper column names.

Q1. Provide the name of the most queried domain.

Right click on the query column header and select Distinct values.

Answer: teredo.ipv6.microsoft.com (with 39,273 requests representing 9.2% of all requests)

Q2. What was the IP address of the machine which issued a maximum number of requests having empty DNS queries?

Select all events by clicking Selection menu and selecting select all events (or just press the a key).

Filter requests with empty DNS queries by right clicking on any value of the query column, select Search for..., paste the (empty) string and click Apply.

The dataset instantly got filtered and only empty DNS queries are now displayed.

Show IP repartition by right-clicking on the id.orig_h column header and selecting Distinct values.

Answer: 192.168.202.78 (with 860 empty DNS queries representing 31.2% of all empty DNS queries)

Q3. What was the IP address of the machine that received a maximum number of NXDOMAIN responses?

Select all events.

Filter NXDOMAIN requests by selecting NXDOMAIN in the Distinct values dialog of the column rcode_name.

Show IP repartition by displaying the Distinct values dialog of the id.orig_h column.

Answer: 192.168.202.103 (which received 7,471 NXDOMAIN responses, representing 14.8% of all NXDOMAIN responses)

Q4. For one of the hosts receiving NXDOMAIN errors, could you figure out some anomalous behavior? If yes, then describe the behavior.

As requests returning NXDOMAIN responses are still selected by previous question, we can directly show the repartition of target IPs and queries by displaying the Distinct values dialogs of both id.resp_h and query columns.

First, we can observe that the target IP 192.168.207.4 is generating 99.8% of all the NXDOMAIN responses (but it is also handling 62.3% of the total DNS requests).

Selecting the target IPs one after another will conveniently filter the distinct values of the query dialog so that you can easily see which domains are being considereded to be non-existant by the target IP.

We can also filter all these 9 target IPs returning NXDOMAIN from the global dataset and observe their respective repartition of NXDOMAIN responses.

Answer: Some target IPs are wrongly returning NXDOMAIN for existing domains. Target IP 192.168.207.4 is returning NXDOMAIN for 18.9% of its responses.

Q5. What was the IP address of the machine that sent the most DNS requests?

Select all events.

Show IP repartition by displaying the Distinct values dialog of the id.orig_h column.

Answer: 10.10.117.210 (which sent 75,943 DNS requests representing 17.7% of all DNS requests)

Q6. What was the IP address of the machine that sent the most reverse DNS resolution requests?

Select all events.

Filter reverse DNS resolution requests by entering the following regular expression in the search dialog of the column query

.*in-addr.arpa$

Answer: 192.168.202.83 (which sent 7,283 reverse DNS requests representing 13.8% of all reverse DNS requests)

Q7. How many DNS zone transfer queries were issued on the network?

Select all events.

Show query type repartition by displaying the Distinct values dialog of the id.orig_h column.

Answer: 440 (which represent ~0.1% all of DNS requests)

Q8. One of the DNS requests querying for a sub-domain of apple.com returned a TXT record which contained a suspicious-looking answer. Identify the connection ID of that request.

Select all events.

Filter DNS requests from apple.com subdomains by entering the following text in the search dialog of the column query

.apple.com

Show DNS requests answers by displaying the Distinct values dialog of the answer column.

Isolate the suspicious-looking answer by clicking on it.

The column uid is displaying the connection ID of the request.

Answer: CmjiklOm3bnHgctw

The dataset 2014-11+DFIR+Network+Forensics+Challenge.zip was taken from the Netresec PCAP page.

Note: questions 1 and 4 were not solved because they didn’t involve any PCAP data.

2. What IP addresses were used by the system claiming the MAC Address 00:1f:f3:5a:77:9b?

Click on the PCAP button of the start page, select the provided HTTP profile, browse the nitroba.pcap file and then click Process.

You should now see the selected fields of the PCAP represented as Parallel Coordinates:

On the listing view located on the bottom of the window, right-click on any value of the eth.src column, select Search for..., paste the MAC address 00:1f:f3:5a:77:9b in the Expressions text field, select Case sensitivity: Does not match case and click Apply.

This will instantly filter the packets to display only eth.src == 00:1f:f3:5a:77:9b :

You can then right-click on the src.ip column header and select Distinct values to display the distinct values with their associated count/frequency:

Answer: 169.254.90.183, 192.168.1.64 and 169.254.20.167

3. What IP (source and destination) and TCP ports (source and destination) are used to transfer the “scenery-backgrounds-6.0.0-1.el6.noarch.rpm” file?

Before loading this PCAP file, you will need to create a PCAP profile to select which fields to import in the application. On the PCAP dialog, select Manage profiles, click on New profile, enter something meaningful like FTP, browse the ftp-example.pcap file to analyse the protocols it contains and select all the fields you are interested in like :

1eth.dst,eth.src,ip.dst,ip.src,tcp.srcport,tcp.dstport,ftp.request.arg,ftp.response.code,ftp.passive.ip,ftp.passive.port

Save the profile and open the ftp-example.pcap file with the FTP profile selected.

Right-click on the ftp.request.arg column header, select Distinct values and click on the scenery-backgrounds-6.0.0-1.el6.noarch.rpm file.

This will display the only packet containing scenery-backgrounds-6.0.0-1.el6.noarch.rpm as its ftp.request.arg field.
Right-click on the row index and select Copy line index to clipboard.

Press the g key and paste the value 34956 corresponding to our packet.

Scroll-back a few rows until you see on the ftp.response.code column the value 227 which corresponds to “Entering Passive Mode”.
We can then observe that the destination IP and port are 149.20.20.135:30472.

Right-click on the ip.src value of the packet containing the file and select Search for this value, this will only keep packets with this IP as source.

Then right-click on the tcp.dstport value 30472 and select Search for the value to add another level of filtering which will keep only packets having our previously selected IP as source and 30472 as TCP destination port.

We can observe that the source IP and port are 192.168.75.29:51851.

Answer: 149.20.20.135:30472 and 192.168.75.29:51851

5. What is the byte size for the file named “Researched Sub-Atomic Particles.xlsx”?

Create a SMB PCAP profile containing the following fields:

1eth.dst,eth.src,ip.dst,ip.src,tcp.srcport,tcp.dstport,smb.file,smb.end_of_file

Then import the stark-20120403-full-smb_smb2.pcap file using the newly created SMB profile.

Filter the packets containing the text Researched Sub-Atomic Particles.xlsx in their smb.file field:

Then display the Distinct values of the column smb.end_of_file

Answer: 13,625 bytes

6. The traffic in this Snort IDS pcap log contains traffic that is suspected to be a malware beaconing. Identify the substring and offset for a common substring that would support a unique Indicator Of Compromise for this activity.

Create a custom PCAP profile containing the following fields:

1frame.time,eth.dst,eth.src,ip.dst,ip.src,tcp.srcport,tcp.dstport,data.data

Then import the snort.log.1340504390.pcap file using the newly created profile.

Displaying the Distinct values of the data.data column and scrolling through the values seems to indicate that some of the data remains constant.

But scrolling the 14,228 distinct values to check if this patterns applies to all of them would make our eyes bleed for sure.
Wouldn’t it be nice if we could instead visualize the integrality of the data content and prove it ?

Alright, let’s do that by splitting the data.data field into the smallest addressable memory units: bytes.
A few lines of Python code will be plenty enough to do that:

1source = squey.source("snort.log.1340504390.pcap")
2data = source.column("data.data")
3for i in range(int(len(data[0])/2)):
4    b=data.view('<U2').reshape(data.shape + (-1,))[:, i]
5    source.insert_column(b.copy(), f"data_byte_{i}")
6str=bytes.fromhex(data[0][8:22]).decode("ASCII")
7print(f"data[4:10]={str}")

Now, it is crystal clear that along all the packets of the dataset, bytes 4 to 10 of the data.data field are always constant:

Answer: bytes 4 to 10 (zero based), which represent the string “ULQENP2” in ASCII.

7. BONUS! Identify the meaning of the bytes that precede the substring above.

There are 4 bytes preceding the static “ULQENP2” string: let’s extract these as a new uint32 integer column using the following Python code:

1import numpy as np
2source = squey.source("snort.log.1340504390.pcap")
3data = source.column("data.data")
4b=data.view('<U8').reshape(data.shape + (-1,))[:, 0]
5b_int=np.vectorize(lambda t: int(t, 16) if t else 0)(b)
6source.insert_column(b_int.copy(), "data_first_4_bytes_as_uint32")

Now, let’s check if we can observe an evolution of this data through time by using a scatter plot.

Wait, it is not evolving through time, it is time.

Answer: A UNIX timesteamp.

The first section is devoted the creation of a parsing file that will allow us to load the dataset. Should you be in a hurry, you can skip straight to the analysis as the file is provided below.

Parsing file creation

Before being able to load the dataset into the application, a parsing file (called format) should be created using the Format Builder tool.

Click on the Create a new format... button located on the start page and then on the Splitters > add RegExp Splitter menus.

Enter the following regular expression into the Expression text field:

(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) "?(-|[^"]*)"? (\S+) (\S+) "?(-|[^"]*)"? (\S+) (\S+)[ ]?"?(-|[^"]*)"?[ ]?(\S+)?[ ]?"?(-|[^"]*)?"?

Then click on the Set axes name toolbar button and copy/paste the following columns name in the text dialog:

date-time time-taken c-ip sc-status s-action sc-bytes cs-bytes cs-method cs-uri-scheme cs-host cs-uri-path cs-uri-query cs-username s-hierarchy s-supplier-name rs(Content-Type) cs(User-Agent) sc-filter-result sc-filter-category x-virus-id s-ip s-sitename x-virus-details x-icap-error-code x-icap-error-details

Note: thoses were extracted from the log file using the following command:

sed -n '4p' Demo_log_001.log | cut -d" " -f2- | sed 's/ /-/'

Loading a file of the dataset will now help you confirm that the data is appropriately parsed but will also automatically detect the types of the columns.

Click on the File > Local files... menus and select one of the dataset log file.

Click on the autodetect axes types located in the bottom right corner of the dialog.

Finally, save the format as squey.format in the bluecoat_proxy_big directory

Note that squey.format is a catch-all name that will automatically match any dataset, and that’s handy for datasets composed of several files. To match a specific dataset, say dataset.csv, name your format dataset.csv.format. If your format name doesn’t match a dataset name, you will have to explicitely select it at loading by selecting Format: custom format and a dialog box will ask you to locate it on the disk.

Loading the dataset

If not done previously:

1. Download and extract the dataset on your machine.
2. Download and and extract squey.format.zip in the same folder the dataset was extracted.

Click on the Local files... button located on the SOURCES section of the start page and select the dataset log files.

During the data ingest stage, it was found that 8,883 rows (~0,11%) were dismissed.
A dialog is displaying them so you can confirm after inspection that they are all redundant headers and no other valuable data was left out.

It took 16.22 seconds to load the dataset (4 text files, ~2.6GB uncompressed data) on an Intel® Core™ i7-12700H Processor and the memory consumption was ~5GB.

It took only 2.96 seconds to reload the saved investigation from disk (~1.1G binary data) as the data ingest stage alredy occured.

Note that in case your dataset is exceeding the available memory of your system, you can always filter out some rows during the data ingest stage by applying custom exclusion rules to specific columns and make your analysis in several steps.

Visual analysis

After loading the dataset, it is displayed in integrality using parallel coordinates.
To gain more knowledge about how the parallel coordinates work in Squey, please ensure to have read the according section of the documentation.

This dataset is composed of 8,130,590 rows and 26 columns and looks like this:

A first look at the displayed axes could already make us discover some facts worth of interest.

Dubious cs-method

Anyone a bit familiar with proxies would probably ask himself why the cs-method column is containing so much different values ?

A right-click on the cs-method column header and selecting Distinct values display each distinct values as well as their associated count.

We can then observe that many of them do not look to methods of standardized protocols but seem rather unusual.

Before going any further, let’s clear our mind from any doubt that this could come from a corruption during the data download or ingest stage.

Here are the SHA-256 checkums of the files composing the dataset:

[/srv/data/bluecoat_proxy_big]$ sha256sum *
db1b8371cbee4a6ff3755c11d5aad81aab4661449b0c5eac14f2f93b9b0b0bf9  Demo_log_001.log
8ff4858536215473e752db29537e46f598e79dd9985f15eedda85115d3c4fd0f  Demo_log_002.log
18759422a0836e61ddc1f2877ba7ae516179f07675a39e2fa8d9bf6baa238731  Demo_log_003.log
27139f8d611686127733e2a0eaf67eaf12e616332bc2cc281ec50af4984518b3  Demo_log_004.log

And here are some of the rows extracted from the original data:

sed -n '981185p; 1070757p; 1151521p; 1168033p; 1252127p; 1516636p; 1570191p; 1613877p; 1817343p; 1843350p; 1844162p; 1844203p; 1844351p; 1847392p; 1847641p; 1847644p; 1861554p; 2185718p; 2561934p; 2648396p; 2747923p; 2962884p; 2962885p; 2963953p; 3019317p; 3023657p; 3164071p; 3165808p; 3172156p; 3329713p; 3382352p; 3591245p; 3709973p; 3741054p; 3743310p; 3745850p' Demo_log_003.log

2005-05-03 16:36:32 3598 45.112.1.58 400 TCP_NC_MISS 9036 4294967282 %03 - - / - - NONE 192.16.170.44 - - PROXIED unavailable - 192.16.170.44 SG-HTTP-Service - - -
2005-05-03 17:30:50 10031 45.116.1.30 0 TCP_ERR_MISS 0 12 7 - - / - - NONE - - - PROXIED none - 192.16.170.44 SG-HTTP-Service - - -
2005-05-03 18:07:26 1 45.116.1.30 400 TCP_NC_MISS 9036 12 %0B%16c - - / - - NONE 192.16.170.44 - - PROXIED unavailable - 192.16.170.44 SG-HTTP-Service - - -
2005-05-03 18:16:30 1 45.116.1.30 400 TCP_NC_MISS 9036 12 %B8X� - - / - - NONE 192.16.170.44 - - PROXIED unavailable - 192.16.170.44 SG-HTTP-Service - - -
2005-05-03 19:01:13 1922 45.14.2.150 0 TCP_ERR_MISS 0 16 |*[%FA - - / - - NONE - - - PROXIED none - 192.16.170.44 SG-HTTP-Service - - -
2005-05-03 21:33:05 459 45.14.3.123 0 TCP_ERR_MISS 0 16 %F9 - - / - - NONE - - - PROXIED none - 192.16.170.44 SG-HTTP-Service - - -
2005-05-03 22:02:36 485 45.114.2.68 0 TCP_ERR_MISS 0 16 %9A%9C%01� - - / - - NONE - - - PROXIED none - 192.16.170.44 SG-HTTP-Service - - -
2005-05-04 00:08:09 1 45.21.4.254 400 TCP_NC_MISS 9036 34 recipientid=100&sessionid=1516 - - / - - NONE 192.16.170.44 - - PROXIED unavailable - 192.16.170.44 SG-HTTP-Service - - -
2005-05-04 00:35:42 9 45.114.3.11 400 TCP_NC_MISS 9036 679 <?xml - - / - - NONE 192.16.170.44 - - PROXIED unavailable - 192.16.170.44 SG-HTTP-Service - - -

As we can see, these were well present in the original data.

This leaves us with at leat two hypotheses.

First, this could be a bug in the proxy OS that could arise under heavy load. In that case, it could be useful to take some mitigation measures like upgrading the sofware, monitoring the CPU usage, reducing the bandwidth usage, redesigning some part of the network or upgrading the hardware.

Secondly, this could be bogus communications of faulty apps that could be reported upstream, or malicious communications that should be further investigated.

Exporting our first discovery

Let’s export this isolated data subset for a potential investigation before focusing on our main analysis.

There is two ways to isolate these dubious cs-method values:

1. Select any wanted values from the Distinct values dialog
2. Filter out valid values by applying a search regular expression

Let’s try the second approach: right-click on any values of the listing in the cs-method column, select Search for..., copy the following regular expression [A-Za-z_]*, select exclude and Regular expressions and click apply:

This instantly filtered our dataset:

Displaying the distinct values of the cs-method column now only shows the selected values:

We can then export the selected rows by clicking on the File > Export > Selection menus.
On-the-fly compression is supported with the following formats: gz, bz2, zip and xz.

The dataset has been exported to dubious_cs-method.csv.xz (5.1KB).
It can easily be loaded back into the application with on-the-fly decompression, headers and columns type autodetection.

Creating our first layer

Note that inverting the selected rows can be done by clicking on the Selection menu and selecting Invert selection (I).
The Distinct values dialog is then automatically updated to display the trusful cs-method values and their associated count/percentage:

This should give us a better understanding of the different protocols at work.

To confortably continue our analysis, let’s isolate the current selected rows in an active layer so that these dubious cs-method values won’t stood in the way anymore. Click on the Selection > Create new layer from selection (Alt+K) menus and give your layer the name you want.

As you can see on the layer stack, our active dataset is now restricted to the rows with trustful cs-method values: we let aside 744 identified rows.

Back to our main analysis

Should there be a bandwidth problem, let’s try to understand how it is distributed across the traffic.

Note: coloring the rows using a gradient of color depending on the size of the requests can opionally be done by clicking on Filters > Axis gradient menus and selecting the sc-bytes axis.

Right-click on the sc-bytes axis title, select New selection cursor and select the top of the axis to filter high values.

We can see which requests are consuming a lot of bandwidth as well as their associated hosts. This is interesting but we are still missing the big picture as lots of contents are not fetch in a single request but with many of them.

It would be nice if we could aggregate values of a selected column by doing a sum on numeric values located on another column. Fortunately we can do this easily using the Sumby function.

First, remove the selection cursors by right-clicking on one of them and select Remove cursors, then click on the Selection > Select all events menus.

Finally, right-click on the header of the c-ip column, select Sum by and then click on the sc-bytes column.

This displays the distinct values contained in the c-ip column, but instead of displaying a count based on the frequency of which they appears, the dialog now displays their consumed bandwidth.

To gain even more information about the supposed types of traffic of each IPs, we can display the consumed bandwidth of each hosts. Right-click on the header of the cs-host column, select Sum by and then click on the sc-bytes column.

What could really came in handy is that selecting a specific IP will instantly filter the dataset and refresh the second dialog, revealing the bandwidth repartition by hosts for this IP.

In this case, we can for example observe than IP 45.114.1.163 has downloaded ~3.43GB from liveupdate.symantecliveupdate.com, which represents 95.9% of its total traffic.

Continue the exploration on your own

Now that you master the basic features of Squey, feel free to continue exploring this dataset to discover additional insights.