Using Zeek for network analysis and detections

What is Zeek?

Zeek (formerly known as Bro) is an open-source network traffic analyzer. The tool sits on a sensor and observes network traffic. It is free, open-source software designed to extract hundreds of fields in network data in real-time. The tool has pre-built parsers for numerous protocols such as (HTTP, SSL, DNS, FTP etc.) and allows for the creation of custom parsers for protocols not yet supported. Zeek can detect anomalies, but not in the same fashion as a traditional IDS (like Suricata). 

The tool mirrors (or SPAN) a router within your network to collect a copy of the traffic. It then takes that traffic and processes, parses and structures the network data based on protocols. The processed data is then stored into various log files (dns.log, http.log, con.log etc.). We will explore a few of these in the below sections to learn how they can be used. A common use of these files is to ingest them into a SIEM platform to craft detections.

Network data. Who cares?

Good network data is key when investigating security events and crafting good detections. To create a complete picture of what's occurring in your network, you must first understand what systems and services are connecting to and how traffic flows through your organization's IT environment. There are many different ways to collect network data. You can collect it via a firewall, netflow or other network analysis tools/technologies. However, a lot of the time, this data is incomplete or extremely expensive to capture. Enter Zeek.

Zeek is a great source of network data for hunting, monitoring and analysis. If configured correctly, it does not boggle down the network or overload security teams with useless data. It extracts fields in network data to provide teams with parsed and actionable information that can then be used to craft meaningful detections to better secure the network. 

Deployment 

1. Get the latest version of Zeek that works best with the system and distro you plan to install it on (E.g., Ubuntu)

• You can find older versions of Zeek on various GitHub pages if there is a deprecated feature you wish to use

2. Before you install Zeek, make sure you have all the required dependencies

• You can install from a pre-built binary package
• You can install the code directly from the source

3. Next, configure the run time environment and define the local networks to monitor

4. Before you can run Zeek, you need to deploy the ZeekControl configurations

5. You can then check the Zeek logs in the below directory to see if Zeek is set up and configured properly. If you navigate to the below directory, you should start to see log files in the directory

• /opt/zeek/logs/current

Get your free course catalog

Download the Infosec Skills course catalog to learn more about these courses — and hundreds more.

Download Now

Zeek logs

Let's take a look at the format of one of the most used Zeek log file. The conn.log as reproduced from Richard Bejtlich's Corelight Bright Ideas Blog.

FIELD TYPE DESCRIPTION

ts time Timestamp of the first packet

uid string Unique ID of the connection

id.orig_h addr Originating endpoint’s IP address (Orig)

id.orig_p port Originating endpoint’s TCP/UDP port (or ICMP code)

id.resp_h addr Responding endpoint’s IP address (Resp)

id.resp_p port Responding endpoint’s TCP/UDP port (or ICMP code)

proto proto Transport layer protocol of connection

service string Detected application protocol, if any

duration interval Connection length

orig_bytes count Orig payload bytes; from sequence numbers if TCP

resp_bytes count Resp payload bytes; from sequence numbers if TCP

conn_state string Connection state ( see conn.log>conn_state)

local_orig bool Is Orig in Site::local_nets?

local_resp bool Is Resp in Site::local_nets?

missed_bytes count Number of bytes missing due to content gaps

history string Connection state history(see conn.log>history)

orig_pkts count Number of Orig packets

orig_ip_bytes count Number of Orig IP bytes (via IP total_length header field)

resp_pkts count Number of Resp packets

resp_ip_bytes count Number of Resp IP bytes (via IP total_length header field)

tunnel_parents set If tunneled, connection UID of encapsulating parent(s)

Source: Richard Bejtlich, Corelight Bright Ideas Blog

This file stores all the logs Zeek created for connection attempts identified in the network. Drilling down into the fields and related descriptions, you can see that this entry shows the connections (TCP/UDP/ICMP) systems in your network are making. This is often a great place to start when learning to use Zeek data and investigating an event. Connection details can often help you baseline systems and identify abnormal or anomalous activity (such as a specific system reaching out to a system it's never communicated with before). 

Other files created when Zeek produces logs are the following:

• Files.log - information on different file analysis 
• Dns.log - additional data on parsed DNS related activity captured
• Http.log - data on HTTP requests and replies captured by the parser
• Ssh.log - SSH connections captured during analysis 
• Ftp.log - ALl FTP related activity detected by Zeek

Zeek produces numerous other log files. You can find more information on them in the Zeek documentation here.

Use-case: threat hunting

Zeek is an excellent tool for threat hunting. Many IDS (such as Suricata) are focused on signature and rule-based detections. Zeek can be used as a traditional IDS. However, Zeek can be used to focus more on specific network protocols for deeper analysis. The more data you have to work with when threat hunting, the better. 

As we learned in the above sections of this article, Zeek stores metadata, which it captures into various files. The files can then be used by security analysts and threat hunters when performing investigations or hunting for bad activity in the network. 

The tool can help analysts:

1. Monitor network traffic flows for anomalous or malicious activity 
2. Profile specific protocols and services (e.g., DNS, HTTP, FTP) to build baselines that can be used to find deviations in patterns (hinting that something nefarious could be happening) 
3. Enrich their security tool data by adding a layer of analytics on their security tools stack

Get your free course catalog

Download the Infosec Skills course catalog to learn more about these courses — and hundreds more.

Download Now

Overview

Zeek is an extremely powerful network analyzer. There are numerous use-cases for the tool ranging from network detection, forensic evidence captures, SIEM alert enrichment and much more. Zeek and an IDS like Suricata have similar methods to produce data and help security professionals defend their environment. They are often used in conjunction to provide a layered approach to security.

In the final article of this series, I will demonstrate how to convert PCAP files into Zeek logs and investigate the data.

Want to learn more? Take my Advanced Intrusion Detection courses in Infosec Skills.