hi i am seth grover i am the maintainer
of malcolm the network traffic analysis
tool suite
network traffic analysis is all about
getting to the important stuff as
quickly as possible there's a lot of
open source and proprietary tools out
there for analyzing raw packet capture
or pcap files
wireshark network miner grass marlin
but analyzing pcapp sets that are large
or from complex networks
with many
tools is difficult
because often
tools that handle raw pcap data
struggle to handle packet capture files
that are you know large larger than
maybe a few hundred megabytes or
gigabyte or two depending on your system
resources
um
so today we're going to talk about
malcolm
a tool that was developed and is
developed at the idaho national lab with
support of the united states department
of homeland security cisa
you may be familiar with some or all of
the tools that make up malcolm because
they're open source and they're all
generally in use
in the
in the security and network traffic
analysis community
but what malcolm provides is a framework
of interconnectivity for these tools
which which streamlines network traffic
analysis and helps you bring all that
important stuff to the foreground as
painlessly as possible
so that's what we're going to talk about
today in this video is using malcolm to
gain insight into both link layer and
application layer network traffic
before we jump into our discussion about
malcolm and some of its primary
components like zeek and the elastic
stack and archime
let's take a minute and talk about
intrusion detection systems so we can
get an understanding of what these tools
do and how they fit into the threat
detection landscape when talking about
intrusion detection systems usually
you're going to be talking about tools
in one of two categories
host intrusion detection systems or hids
utilize a native agent that runs locally
on individual hosts and endpoints on the
network
and these agents monitor you know not
only
maybe network traffic and stuff at the
device nic level
but also track modifications to system
files or monitor user authentication
events configuration changes
and and report these events to a central
manager for alerting and reporting
host intrusion detection systems is not
what we're going to be talking about
today as it's not really the main focus
of malcolm there are plugins that you
can use to get host data into malcolm
and maybe at some future point we'll put
together a
a video that will instruct you how to do
that
but for now we're going to be talking
about the other category of ids which is
network intrusion detection systems or
nids
network intrusion detection systems are
generally passive and out-of-band
programs or devices that capture and
analyze network traffic at strategic
points in your network
in order to monitor traffic among
devices in the network or between
devices on the network and the outside
world
so um a couple of different ways that we
can do this the monitoring and analysis
can be done concurrently in other words
analyzing the traffic and forwarding it
along
sorry capturing the traffic and then
forwarding it along for analysis as it's
captured
or um
the other way is the network traffic can
be analyzed
after it's been captured in the past
right
we can capture it first with with
whatever set of tools we want um
matt we're going to talk mostly about
the latter today uh just for our
examples i'll be importing pcap data and
then analyzing that pcap data rather
than using a live network sensor to
capture and forward that data the
malcolm project does include a
linux distribution called hedgehog linux
that's kind of a stripped down debian
based linux that has
these capture tools and an easy
configuration for those capture tools to
set them up and capture and and forward
to malcolm
there's another one of the videos uh on
the malcolm network traffic analysis
youtube channel that talks about how to
set up hedgehog and malcolm and set up
that forwarding together but today we're
going to talk about um more of the i've
already captured i've got some pcap and
i want to upload and analyze it with
malcolm
one other good important point about ids
is it's generally passive meaning that
it shouldn't alter the network traffic
itself as a side effect of its analysis
there are systems out there that
actively drop suspicious network traffic
and those are called intrusion
prevention systems
or ips instead of intrusion detection
systems or
ids and it's important particularly in
networks with critical infrastructure
one of the focuses of malcolm is
industrial control systems or or
critical infrastructure networks with ot
protocols while it's perfectly suitable
for using in a purely it network that's
fine
but particularly on ot networks it's
important that we don't go knocking
things over
as far as network services go because
because that may be critical
infrastructure right
and so it's important that ids is done
out of band and and
in a way that won't influence the
network traffic itself
um so let's talk about some of the
different approaches to ids uh each has
its strengths and weaknesses i guess you
could say
first there's signature based
uh detection and this is what you're
familiar to in the context of like a
antivirus
an antivirus program you know it has or
a malware program that recognizes
malware based on some pattern or
signature
uh in the malware itself and then
you know it has a list of predefined
patterns that it's looking for
and when it finds a file or traffic or
whatever that matches one of those
patterns
it flags an alert as malicious behavior
while signature based ids is great for
known attacks and it's usually efficient
as far as resource utilization goes
it's not generally effective at
detecting novel attacks
because it's like the new attacks the
zero days or whatever you want to call
them they're the unknown unknowns and
you you can't write a signature for
something that you don't know about
generally
the the second camp there for ids is
statistical anomaly based detection
um
this is this is
the machine learning right machine
learning is
either math cleverly disguised as magic
or magic cleverly disguised as math i'm
not exactly sure which it's cool um but
uh the basic idea is that it's creating
a baseline for trusted network behavior
and then it will compare new behavior
against that baseline um using this
magic or math
and this technique can be detec it can
be effective in detecting anomalies or
um you know novel attacks things that
you're not necessarily looking for but
uh
you have to it can suffer from high
false positive rates especially if you
don't have
the um
the baseline there to to really define
what is normal which can be difficult in
a network assessment particularly if
you're coming in from the point of view
of like
i don't necessarily maybe have this
long-term um this long-term network
traffic i want to analyze i just have
some pcaps or whatever right
it can be computationally expensive as
well
as far as resources for the um
the
anomaly the machine learning kind of
anomaly stuff
certainly has its place in the threat
detection landscape and
there are tools built into malcolm once
we won't really be talking about today
necessarily but
maybe in a future video we'll do one for
that as well where we can set up anomaly
detectors and
and start to get that baseline
established but
today we'll be talking more about i
think the third camp which is where um
i feel is a nice kind of blend of of
knowing what i'm looking for versus
trying to
flag interesting things that i might not
be looking for and that is stateful
protocol analysis
stateful protocol analysis-based
detection uses knowledge of network
protocols to look for deviations from
profiles of generally accepted
definitions of normal activity in other
words you know i know what normal http
http traffic normally looks like
you know usually i see
these error codes and these kinds of
requests or you know i know what smb
file shares do most of the time and so
if i can kind of categorize based on my
knowledge of that network protocol um we
can
programs can then say hey you know
here's a summary of what we're doing in
this particular protocol or that
particular protocol and it helps us
recognize patterns as we look at that
data
so what type of intrusions or attacks
might we hope to uncover using an
intrusion detection system
let's talk about that a scanning attack
first a scanning attack is
used to
detect network topology to assimilate
information about a system or a network
being attacked
by attempting connections to a range of
ip addresses within a network or
scanning for open ports
which would correspond to corresponding
to responding services on those hosts
an attacker puts together a map of the
topology of your network the types of
network traffic that are allowed through
a firewall what active hosts are on the
network the operating system kernel and
software versions running on the exposed
services
this information can then be used to
launch attacks aimed
at at specific vulnerabilities with
specific exploits
and and a good idea should be able to
notice these kinds of attacks these
kinds of accesses because they might
possibly be seen as a series of
sequential connections from one host to
a range of ip addresses or ports or you
know a brute force attempt to to log on
to some exposed service
and then you know can alert on that host
scan or that port scan that took place
a denial of service attack
works by flooding a network or host with
an overwhelming number of connections or
requests so this could be something as
simple as sending a large number of ping
packets which is like a ping flood or by
forging the initiation of a tcp
connection a syn flood causing the host
to be
unable to respond to legitimate
connections
intrusion detection systems are usually
good at categorizing traffic
from or to a particular host or service
and so um can often track things like
connection state for various network
protocols or at least you know the
number of connections attempts which
makes identifying
a denial of service
attack pretty easy um it's pretty it's
pretty obvious i guess when it's
happening that that your network is
overwhelmed or your host is overwhelmed
um just based on the pure volume
and then finally a penetration attack
this is any type of attack which gives
an unauthorized attacker
the ability to actually access system
resources privileges or data by
exploiting a misconfigured system or a
software flaw
these types of attacks are more
difficult to identify because often they
they
look like legitimate traffic right or
they may be exploiting
like i said a misconfiguration or or
some unknown loophole uh to get in and
get a foothold in the system
and so it's at that point once they get
a foothold in a system and can maybe
pivot to another area in the system
it's easier for them to cover their
tracks for future communications and
mask their commands as normal network
traffic so this is particularly true
with custom targeted zero-day exploits
or
exploits for which the attack vector was
not previously known
and hence there's not signatures to
detect them
however ids's can still be valuable in
identifying penetration attacks when
they are protocol aware allowing
analysts to recognize changes in
patterns of behavior or unusual
operations in the context of those
protocols within normal network traffic
so um zeek we're going to talk about
zeke for a little bit zeke which was
formerly known as bro
is one of the two
pcap analyzing engines used by malcolm
to generate metadata about network
traffic which metadata is indexed and
made searchable through malcolm's
visualization tools
so
before we get into what those tools are
let's do a little more laying the
groundwork of what zeke's capabilities
are and what zeke is to better
understand what it offers
analysts as a malcolm data source
so where does z come into the picture
what is zeke
sometimes it's referred to as zeek ids
and it incorporates some techniques from
the previous slide
but zeek is more than just a
intrusion detection system really it's
it's it's really great it's an
extensible open source passive network
analysis framework and
it does packet capture trafficking
inspection intrusion detection
records flow logs it can even be used
has a robust scripting and data
structure framework for log enrichments
or for
writing your own you know creating your
own logs or even writing your own
network analyzers
so if i had to categorize zeek into one
of the three detection method categories
from from
our previous slide i'd categorize it in
the
stateful protocol analysis detection
camp
zeke's network traffic parsers examine
network traffic at the application layer
and then reports on the behaviors of
hosts that are communicating over those
protocols so these logs can then be used
to do a more in-depth
manual or automated analysis as we'll
see throughout our discussion of malcolm
today
i would say that zeke as well is
fundamentally different from other ids's
in that it goes beyond pure signature
matching
in favor of uh of analyzing that
application layer behavior of the host
themselves although it does have
signature
matching capabilities similar to
yara or snort
generally its focus is more on
just
parsing network traffic at the
application layer
zeek features can be combined with
combining powerful ways to provide
insight into network traffic
with ziklog the network analysis
can include content extraction for
example extracting
exfiltrated files from pcapps for
further examination
behavior analysis and session
correlation as zeek is is highly
stateful extensively tracking
application layer network state it can
be used to determine what else took
place during a session or during the
communication between two hosts or maybe
what proceeded or followed a suspicious
event and then like i said zeek is
extensible
support for uncommon protocols for
example ot protocols can be added via
scripts and plugin architecture
as as i work a lot with analysts that
deal as i said earlier in industrial
control systems or
critical infrastructure kinds of
networks a lot of these protocols are
not commonly seen you know on the
internet as a whole and so a lot of the
times off-the-shelf tools that you'll
find
for dealing with um
don't include dealing with this kind of
traffic and so
one of the things that malcolm has done
with its use of zeek is
is add a bunch more support for ot
protocols for ics protocols and and then
allowing us to kind of bring that
metadata that's associated with those
communications to the forefront
alongside the you know more common i.t
protocols
so zeke is really
powerful tool it's commonly used in
network traffic analysis but it does you
know have its own set of hurdles and in
a minute we'll hopefully see how malcolm
you'll hopefully see how malcolm helps
overcome those hurdles
while zeke
for example you know zeke
is going to give us that metadata but it
can be difficult for for someone who
is not you know an expert traffic
analyst an expert network analyst to get
back from the zeek data to the original
packet payload
um
and and you know if you do need to get
into the to the actual low-level payload
um
it can be difficult to like go from zeek
to say wireshark and open that up
um
with just the the tools that you might
be used to using um the one of the other
things right zeke it
generates a bunch of flat text log files
right a bunch of delimited or json log
files and if you've got gigabytes of
text files um there you know there are
some tools to to manipulate those and
stuff but again if you're not um maybe
uh uh really well versed on the command
line or um if you don't already have a
tool set in place to process those text
files it can be difficult to to run zeke
and then know what am i supposed to do
with all of this
all this logs all these logs
um
so uh we're going to talk today about
zeke mostly in its context as a
component of malcolm
to perform post capture network analysis
against pcap files that we've already
gathered previously um
so uh let's talk about the kinds of logs
that zeke generates so that when we get
to looking at that data we can recognize
what what we're seeing
first uh the
kind of the backbone of zeek traffic
analysis is con.log
con stands for connection and it's
that's what it is it's network session
tracking
con.log is the backbone of a zeke
analyst
of a zeke analysis because each line of
con.log
each record in con.log represents a
unique network session which is
identified by a four tuple consisting of
uh originating or source you can kind of
think of it like that uh source
or originating ip and port and
responding or destination ip and port so
originating ipport destination iport
makes up that for tuple that is a unique
identifier for that session so each
connection in con.log each session is
assigned a randomish 18 character unique
identifier or uid
and so a particular session's uid
from con.log
will be referenced in any other zeke log
files generated from that same network
traffic so for example in the case of a
an http session between a web browser
and a website there may be one line
representing the entire session in
con.log because that http session is is
a tcp connection and it's from an ip and
a port to an ip and a port and then
during the course of that connection
there could be many http operations
gets and posts and
responses and requests and each of those
actions in the http section each of
those operations will be represented in
http.log
so you may have one line in con.log with
this uid
and then that uid will tie to many lines
in http.log
because those requests belong to the
same session
and so you can use that con
dot log uid that zeke uid to to find out
like what happened across all of my
network traffic in the context of this
session
so there are a lot also besides con.log
of protocol specific log files that zeke
will generate so taking note of which
log files are generated from a network
trace
can can give you insight into what's
in your network
even before you begin analyzing the
file's contents for example if i see an
ssh.log
in a network maybe where i wasn't
i didn't know i had ssh going on or
something like that right
that can that can be like hey i better
go look at that because i don't i don't
know what that is and so um even just
starting it like what log files do i
have what does zeke generate today on
this pcapp can give you a good idea of
just what traffic is in your network
zeek has a really powerful file analysis
engine that attempts to detect and
identify when file transfers occur so um
in other words any time a file is
transferred across one of the protocols
that zeek can understand
zeek can recognize hey this is a file
transfer this is a file being uploaded
or downloaded across your network using
you know http or ftp or smb or irc
or
smtp email
there's various different protocols that
are supported in zeke's
file analysis engine
similar to connections each file in
files.log is assigned a random file
unique identifier or fuid that can be
referenced in other log files so
very similar to our previous example a
file is transferred by
an http request and response
and um
that file will generate a line in
files.log with a unique fu id
and then
that could be referenced in the http.log
to see the details of that connection
right so use that uid just in the same
way that you use the uid and con.log to
reference the session across the entire
analysis data set use that files.log
fuid to reference that file transfer
across the various different protocols
that it may have been involved in
entries and files.log can also be linked
to the sessions during which they were
transferred in con.log by the connection
uid
so this these two file fields that
con.uid
and that files.fuid
can be really really important for
getting context about what's going on in
a network session
two specific types of files
that zeke breaks out into their own log
files is uh pe.log pe stands for
portable executable and this contained
entries about
file transfers that were done for
portable executable files in other words
you know a windows.exe or a linux elf
file
that might be specifically of interest
as a network
security analyst because oftentimes you
know
you may be like hey you know you we're
not supposed to be downloading
executable files or whatever and running
them on our on these systems on these
endpoints
there's also x509.log which contains
information about
x509 formatted public key signatures
or public key certificates like you
would see in an ssl
or tls
session
one really important log file for zeek
and and almost kind of where i would
start if i was going to look at a zeke
log uh
a collection of z clouds would be
nodus.log
notice.log is zeek's concept of an alarm
a big red flag a way to draw extra
attention to an event
so notices can be um generated from any
other zeek script or protocol as its
processing traffic
zeke currently implements i think
maybe 50 or so notices kind of in its
default configuration and then malcolm
adds several more
ranging from um
you know brute force ssh login attempts
to sql injection attacks to expired ssl
certificates and then some of the ones
that malcolm adds for um
recent cves that that you may be aware
of or uh have heard of in the in the
news in the last couple of years
events that are
categorized according to the mitre
attack framework um
so when you see the presence of a
nodus.log you know go we'll go check
that out and say
what's going on here that the zeke
thinks this needs to raise this red flag
that i that i should look at this
traffic and we'll give some examples of
those as we get into the um
the
demo kinds of stuff here which
uh my plan is to sprinkle throughout
this video so as you're watching this
video on youtube
or uh if you know wherever you got it um
that's that's kind of my plan is
there'll be there'll be some demo stuff
in here as well
uh weird.log weird is um a good place to
begin when looking for
anomalous
network traffic from the point of view
of
anomalous
in that the protocol itself is not being
is not behaving the way that you expect
it to um you can kind of some people
kind of look at weird as like a notice
log light like stuff that's
kind of strange or maybe not
you know a little bit out of the
ordinary um but
it could be you know it might be nothing
um and so you know it's something to
look into but also you need to
understand your network in order to um
you know to maybe weed out some of the
false positives or things that you you
know actually do expect to see your
network for example you know what
because what's weird in one network
might be perfectly normal in another um
for example in in some networks that
i've been in and seen where old serial
protocols are being encapsulated over
ethernet
um that that may be very common in uh
an aging industrial control systems
network
uh that's using you know
serial modbus or something like that um
and then
you would see from zeke a weird.log
that's like hey i've got non uh
iptraffic over ethernet
um
and
i look at that in an ot network and i'm
like yeah you know this is pretty much
what i expect that's not really that
weird but in maybe in your it network or
in your corporate side of your network
um
that would be unusual right so uh
i don't one man's trash is another man
treasure or whatever
but you just kind of need to get
familiar with your network enough to
know really what is weird and what's not
signatures.log
is used to flag kits from zeek's
signature based engine
and is also used by malcolm to log hits
from file scanning engines on
transferred files extracted by zeke and
we'll talk more about that later
um and then this one uh we won't spend a
ton of time on this but um
at a configurable interval uh defaulting
to one day
zeke will dump summary lists of various
entities endpoints services whatever
that it's seen over the course of that
period so that might include ssl
certificates mac addresses um
hosts that have performed tcp handshakes
modbus
servers and clients and tcps uh i said
tcp services but um
yeah like
you know what services what servers are
on your network
knownhost.log along with con.log
can be an essential part of zeek for
building a network diagram or from from
validating an asset inventory list
zeke may also generate a software log
software.log where it can identify
software communicating
across the network client server you
know and
possible if possible include the actual
version of that software so examples
might include identifying windows
operating system versions and clients
and servers
communicating over http ftp ssh smtp
mysql
and this can be useful during an
assessment to identify
network hosts or devices that are
running software or firmware with known
vulnerabilities and uh
or or software that's out of date right
that hasn't been patched uh and when
identifying servers by operating system
type of application running and and you
know like i said the version of that
software
um
so we've talked a little bit about zeek
now let's talk about archime
archime is the other pcap analyzer used
to populate malcolm's network session
metadata database
um so archime
has a lot of similarities to zeek in um
in that it parses network traffic data
and and generates these these sessions
these these logs
that represent network connections
these archime session logs are written
into an elasticsearch database where
they are indexed and they become
searchable
what's unique and powerful about archime
is that
these network sessions then can be tied
back into
that original payload that exists in
that pcap file so that allows for deeper
packet inspection and searching that's
not just limited to
packet headers
so
that's kind of the list of components
that put together that are put together
to make malcolm
at least the main ones
the components comprising malcolm are
our industry standard open source tools
and uh that makes it easy to integrate
malcolm with other solutions in those
tools respective ecosystems whether that
be importing more z plugins or or
dashboards into kibana or whatever it
happens to be
i've got a list here of the network
traffic
network application protocols that
malcolm can can parse interpret
and and there's dozens of them there as
you can see including several protocols
commonly seen in ot networks
much of malcolm's development right now
is dedicated towards improving malcolm's
coverage of protocols used by ics
devices
um so let's kind of talk about the the
journey that your pcap file will take as
it uh is on its way to being enriched
and indexed and user searchable
um upon upload malcolm generates the
metadata for
the network
traffic that was represented by that
pcap file using both zeek and archimese
molec capture
and so that pcap gets sent two
directions
um
mullick capture aggregates its metadata
for particular network connections and
it aggregates that into a session record
which is then written into elasticsearch
for us to index
zeek as well as we've talked about
generates these log files that are
broken out primarily by application
protocol and they similarly contain
uh metadata that that is you know quite
that is not unlike that done by mullet
capture
malcolm also uses zeke's ability to
carve out files transferred over these
protocols and these files can be scanned
for example by an anti-virus tool or or
preserved for analysis with
you know with other tools with whatever
your your tool of choice is
so those z clogs are forwarded by
filebeat to logs dash for further
enrichment and it's normalized to the
same underlying database schema the
underlying the same underlying field
schema that archime uses so that those z
as much as possible those z logs and
those arcaney sessions can be viewed
side by side as apples and apples
and then that's index into elasticsearch
as well
and then once ingested by elasticsearch
malcolm provides two interfaces for
visualizing that network traffic
cabana
primarily for the z-clogs and then
archimede viewer which can be used to
visualize the z clogs as well as the
arcane session data
so now that we have had a kind of an
overview of the main components of
malcolm and how they fit together and
kind of the theory behind what they do
let's get into the process of actually
um
doing network traffic analysis and and
that
surprisingly that doesn't start with
just uploading your pcap file you would
think that would be the first step but
there's actually one very useful step
that we want to do first as much as
possible
and that is to identify network hosts
and subnets
so
for that there's an interface that
malcolm provides
called the host and network segment name
mapping interface
and that allows you to assign names for
network segments
or or subnets and a host and that might
be any kind of endpoint whether that's a
a server or a desktop or a laptop or
a plc or an hmi or
you know whatever whatever has ip
addresses or mac addresses on your
network
as z clogs are processed into malcolm's
elasticsearch instance the log's source
and destination ip and mac address
fields which is
zeek.orig underscore h and that's that
naming convention we kind of hinted at
it earlier but that originating host
that's a ridge underscore h
zeek dot resp underscore h that's
responding host
and then the mac address fields a ridge
underscore l2 underscore adder and resp
underscore l2 underscore adder
so basically you know source ip desktop
source mac and desk mac
they're compared against the lists of
the host addresses provided in this
interface and uh when a match is found a
new field is added to the log it's zeek
dot a ridge underscore host name or zeke
dot resp responding underscore host name
uh so that a ridge underscore host name
and the rest under score host name is
added to allow your custom defined host
name that you've you know mapped out to
these ip addresses or mac addresses to
be
to be actually written along with those
logs
for traffic matching the list of segment
addresses provided zeek dot a ridge
underscore segment and z.resp underscore
segment fields are added
if both seek.origin underscore segment
and zeek.resp underscore segment are
added to a log and if they contain
different values different subnets then
a tag a value to the tags field will be
added with the cross segment value
and that allows you to conveniently
identify cross segment traffic
so this you know maybe right now you're
like my eyes are glazing over what is
this resp underscore a ridge underscore
l2
um it's it's really it's not that
complicated it's just a way for you to
say hey
here's some ip addresses i want to give
them names that i recognize here's some
network subnets i want to give them
names that i recognize right this is my
this is my corporate zone this is my ot
zone this is my dmz
whatever um and then
if you can identify what those are ahead
of time upon ingestion of that traffic
will just malcolm will tag that traffic
as such and then note when things like
cross traffic
cross-segment traffic is happening right
if you have device in your
control systems network that is that is
talking to reaching out to the internet
or talking to a device in your corporate
network uh you know it would
ideally flag that as cross-segment
traffic and you would be able to to
recognize that without having to go hunt
it down yourself
there's all field in this in this
interface called the
required tag field and basically what
that is is it's a way for you to say
only
apply this tag or only apply this name
this segment or hostname if this tag
exists
as we get into uploading pcap files in a
minute we'll talk about how you can
apply tags that go along with your
uploaded pcap data so in other words um
if if for example i'm uh uploading a
pcap that's represented by a particular
representative of a particular um
facility
and that facility's name is you know
facility abc
i could tag that upload with facility
abc
and then in this
network segment name mapping interface
basically say only apply this name to
this ip address if the tag facility abc
is present
these mappings can also be defined as in
a delimited format in cider dash map dot
text and host dash map dot text in the
malcolm installation directory i would
refer you to the malcolm documentation
on github to see the format for that i
hope to be able to come up with a
feature in the future where like you
could take an excel spreadsheet or
something similar and upload it straight
into this interface and have it
magically figure stuff out for you
it's not quite there at this point but
um that's that's something that maybe we
can look at doing in the future
so we're going to go ahead and do our
host and network segment mapping now and
i'm going to do that by navigating on
the malcolm
web interface to
my malcolm instances running on
localhost and the name map ui
is the interface that i'm interested in
and since i was previously authenticated
before i started recording this i
i didn't have to put in my username and
password
now if i was going to start doing this
from scratch um
you know depending on how big my network
is
i'd either just start typing i p
addresses in you know one at a time
something like this
then click save
or maybe another segment
home network
and we're going to call oops 1 two
16.0.0.12 or something like that
and you know this is home network etc uh
in this case for our example i've
already created a network mapping for
the pcap that i'll be using for our
for this demonstration
so if you've done that before you can
actually save that and back that up and
then restore it later which is what i'll
be doing now
so i'm going to click import
and i'm going to replace the name
mappings with this netmap.json file that
i previously created that already
corresponds to this pcapp
i'm going to do that and you'll see that
it
populates my list here with segments and
hosts
that
that make up my network map um
as you're doing these you can also uh
you can search the list for example if
you want to come back later and
look up you know what you what you
created as your um you want your list of
you know historians or something like
that i can start typing historian here
and it'll filter that list
or
just any text that's available here
you know we can start typing that and
then it will filter that list but once
we've defined our host and network
name mappings we're going to go down to
the bottom of this list
we're going to click save
and click yes to save our name mappings
and then what we're going to do is
before
that will take effect into our name
ingestion into our pcap ingestion excuse
me
we will need to click restart logstash
and it will tell us
make sure that we actually want to do
that want to apply the save name
mappings and restart logstash
and if we do that
it says log stash ingestion is
restarting in the background log
ingestion will be resumed in a few
minutes
and i can click ok
and
and at this point after i wait a couple
of minutes and wait for logstash to come
back up we will be able to
to continue with our upload and make
sure that those ip addresses are mapped
to the names that we specified here
we've defined our network uh
subnets and host names um we're ready to
upload that pcapp data um it
elasticsearch is a write once read many
kind of mentality of a storage um
platform and so we we can't upload pcapp
data and then go back after the fact and
apply those network and host names
we need to
um we need to have that ahead of time so
that we can enrich that data as we're
doing it and so if you come up later and
you say oh i didn't do my network hosts
or whatever
it's not like a huge deal
malcolm generally is pretty quick to
analyze network traffic and so you know
what i would do is is wipe that data out
clear the database for malcolm and then
you know apply my network
subnets and host names and just reingest
the data is kind of the workflow that we
generally see
so so once we've done that malcolm can
be must be provided with network capture
network traffic to analyze right in
other words a pcap file um
so we've talked a little bit how that
could be done with dedicated network
sensors like hedgehog linux
oftentimes though in an assessment
you'll
be given pcap files that have been
previously captured and and you know
provided to you as network security
analyst or that you have captured at
some other point in your network or some
other point in the past and you're
bringing in to analyze now
so pcap files can be uploaded to malcolm
by processing
for processing by accessing the the
upload interface so whatever your
malcolm ip address slash upload on the
host at which malcolm is running
prior to starting the upload as i
mentioned you can add tags which will
allow the data from the pcap files
being uploaded to be searchable using
those tags later on
there's also some other behavior here on
on how the pcap file is parsed uh with
regards to whether you're analyzing it
with zeek or whether files are being
extracted or not generally i like to set
these in the um the configuration
options so they're just done globally
and i don't have to mess with them here
but if you want to override the global
behavior that you've set during
configuration with as far as the zeek
and zeek file extraction goes you can do
that here in the upload interface
we're going to upload our pcap file now
by navigating to the
localhost
upload
and that's going to present us with the
malcolm capture file and log upload
log archive upload
interface that we that we just discussed
i have got my pcap file here
and so as it uploads this pcap file i
talked about the tags it's going to
apply
so um you know if i wanted to tag this
with
for example i'm doing this for uh for
this
uh training video so i could
create a tag called training and it
would add that or if this was a
particular site you know um site abc
customer123
uh you know incident
omega
whatever oops you got the idea anyway i
could apply whatever tags i want here
and those tags would be
searchable after i do the upload
additionally malcolm will take
and create tags from the name in the
pcat file itself so in this case where
the pcapp's file name is cyberville.pcat
the tag cyberville will automatically be
applied
so i'm not going to create any
additional tags here i don't really need
any besides this default one that i've
got
from the file name
but i could add as many other pcap files
as i wanted to here if i had multiple
pcap files
and and then once i'm done i can either
start them individually or just click
start upload
before we we take a harder look at
cabana and archimese user interfaces
let's talk for a moment about the fields
that logstash can use to enrich log data
before it's written into the database
so um
in other words there's there's a lot of
stuff that we can infer from our network
session metadata that might not
initially be in that data
mac addresses for example mac addresses
can generally be mapped to a hardware
vendor as
the first three
octets of a mac address
are called an oui or organizationally
unique identifier
it can be used to distinguish a network
card that was manufactured
by you know intel from one manufactured
by dell or you know a vmware network
interface or
an interface on a schneider plc or
whatever
this uh this mac address then is used to
look up that vendor and and that's added
into that log whenever possible
malcolm also can be configured to do
guip and asn lookups for ip addresses
we can identify internal and external
traffic based on ip ranges or in other
words you know private ip ranges versus
publicly routable globally routable ip
addresses
malcolm can do reverse dns lookups it
can do
dns query and hostname entropy analysis
in order to detect dgas or domain
generation algorithm host names that are
often used by malware so um some malware
will will use
for um
it's you can go read it on wikipedia or
whatever right look up domain generation
algorithms uh but basically you know the
malware generates this long giant
uh
hostname
or uh
url or whatever that that is like
a bunch a whole a whole bunch of
characters letters and numbers dot a
whole bunch of ugly stuff
and uh
that high entropy that that randomness
in that um
url host name uh or that dns lookup or
whatever uh can be used to uh can be can
be bubbled to the top for your for your
analysis
the other kind of things that we'll do
on enrichment with malcolm is community
standard fingerprinting algorithms to be
applied
whenever applicable so that can make
malcolm's data
cross-referenceable with other tools one
really good example of that is
a flow hashing technique called
community id
that
basically takes the the
relevant stuff in a network connection
ip addresses and ports
primarily
and generates this unique hash
of that flow and then that hash can be
used to cross-reference
logs from you know not only inside of
malcolm zeke and arcane but you know off
to surakata or or whatever other network
tool
that supports community id and it's
quite a long list
the other one that i would mention we've
already kind of talked about this is
enrichment is done
to populate that tags field with each
log
and some of the tags that i would make
particular note of would be these ones
that are listed here the internal source
internal destination external source
external destination those are hopefully
self um
you know self describing or whatever but
uh to identify traffic that is on a
private subnet your 192 dot your 172 dot
your uh so your 192 168 dot your 172 dot
and your 10 dot
ip address ranges versus globally
routable ip addresses
and then that cross segment one if
you've defined your network segments in
the
previous interface that we talked about
that cross segment tag will be applied
whenever it sees traffic in the z clogs
that crosses one of those network
segment boundaries
once you've got data into malcolm it'll
take once it's been processed after a
few minutes you'll start to see logs
trickle in and and you can visualize
that log data
cabana is one of malcolm's two user
interfaces for visualizing log data
where cabana really shines for me is in
providing
a
an intuitive interactive representation
of log data that um
that simplifies the process of starting
at like this high level
and and then being able to drill down
quickly to stuff that is interesting to
you so filtering from this big pile of
hay and you're looking for the the
needle in the haystack to like a much
smaller pile of hay
that makes it easier to find the needle
being able to quickly drill down on an
individual host or connection of
interest
malcolm comes with dozens of pre-built
visualizations specifically for data
ingested from uh from zclogs its
dashboards fall into two categories
overview dashboards and protocol
specific dashboards and we'll review
some of those here in a minute
and a site from the pre-built dashboards
cabana provides an easy
drag-and-drop wysiwyg editor for
creating um new visualizations on the
fly
now um
the key to effectively using kibana is
learning how to
how to apply filters how to apply
filters how to how to search the data um
so
as as we talk about applying filters
like
you know you'll be using these patterns
throughout all network traffic analysis
in cabana
to do what we just talked about to start
from this big picture and drill down
into something to zoom in on something
of interest
the first step to applying filters is
identifying the time range of interest
this can be done by using the time
filter controls that are in the upper
right hand corner of the interface
if
and this is me as much as anyone else
i've ever seen but
oftentimes if i'm not seeing the traffic
that i want to see
and i'm like where's my data it's
because i'm searching on the wrong time
frame right it's because i captured my
pcap two weeks ago
for analysis and i'm finally getting
around to it and like the default
controls for cabana are are like set for
the last 15 minutes or something like
that so always check your time frame in
that uh in those time controls in the
upper right hand corner and then you can
use that time histogram that that that
like logs over time graph that you'll
see on most of the kibana interfaces
most of the dashboards to to zoom in or
zoom out based on what time frame you're
interested in
additionally the query bar allows you to
specify search constraints
and you can do that using a couple of
different syntaxes actually
the traditional lucene query syntax
or
the new
kql cabana query language syntax and in
the documentation and in some tables
that i'll probably include here in this
video you'll see um maybe some of the
differences in those syntaxes and and
there's really no magic bullet other
than just you know having the reference
bookmarked and uh and then getting used
to what that syntax is for creating your
queries um modifying the contents of
this search bar
and then hitting enter or clicking the
search icon to the right of it will run
the search and update the results that
are displayed so um
you know the uh time filter that search
query bar and then finally the filter
bar which is a more
gooey kind of based filter creator
underneath the search bar is the third
way of specifying search constraints
although it provides more of a gui kind
of interface to do so
it's generally not quite as like free
form or flexible as as writing textual
queries
in most cases there's not really a
meaningful distinction between putting
query terms in the query bar versus the
filter bar
what the filter bar does allow you to do
which is nice is allows you to pin those
filters so that when you navigate from
one dashboard to another
those filters stick around and the ones
in the query bar generally don't
filter bars are also what's the filter
bar is also what's populated when you
click on values in the charts or tables
so um whenever you see a value in a
chart or a table in a cabana
visualization you mouse over it there'll
be like a little
magnifying glass with a plus icon or a
minus icon in it
so you can use that to
create or exclude filters
or create filters for or filters to
exclude those values based on what you
want to see or don't want to see in that
result set and as you click those inter
click those magnifying glasses you'll
see the filter bar is updated to
to reflect that filter
i've read that a future release of
kibana will merge the query bar and the
filter bar
into like one component i i'm i'm not
exactly sure how they'll do that but i
guess that's something we have to look
forward to
let's talk about the overview dashboards
the dashboards under the general section
of malcolm's cabana interface
provide
a high level overview of network traffic
from across all of the
logs generated by zeek in other words
they're not restricted to like a
particular
application protocol
these dashboards are a good jumping off
point for investigation when you're
trying to get a feel for the network and
application protocols
and and the hosts that are on that
network and using those protocols
uh one of note is the notices dashboard
as discussed earlier zeke notices are
the tools way of raising some uh some
event to the forefront of an analyst's
attention
and those notices are are summarized
here in the zeke notices dashboard
the third party zeke plugins that
malcolm uses
to generate additional notices or to to
analyze traffic in different ways can be
found in the malcolm readme on the
malcolm's github page
they include some of the interesting
ones include but aren't limited to
notices that are generated for
clear text passwords detected in http
post requests
non-compliant http post requests
sorry non-compliant http requests
like those that might be used for um
you know for http smuggling um extor
obfuscated file transfers
um i mentioned this earlier but behavior
or techniques categorized according to
the miter attack framework um and then
like lots of various other cves and
vulnerabilities uh bad neighbor call
stranger
um cigred zero logon
uh
you know the ecc certificate validation
and the
new tls unencrypted session ticket
dissection
um the eternal family of samba windows
exploits from smbv1
which includes eternal blue internal
synergy eternal romance
uh double pulsar
uh
other various smb exploits uh ripple 20.
so
the developers of malcolm
endeavor to stay abreast of of
developments in the threat landscape and
whenever possible we release updates
that include you know new zeke plug-ins
to detect
uh or scripts to detect these
vulnerabilities and exploits as they are
discovered
another interesting dashboard from a
security standpoint
a pair of dashboards actually is the
security overview dashboard and the ics
slash iot security overview dashboards
these highlight events that may be of
particular interest from a security
standpoint including zeek notices
signatures triggered from file scans
clear tracks transmission of passwords
outdated or insecure versions of
application protocols
traffic originating from or directed to
public ip addresses
types of files transferred and more
these these dashboards can be a good
jumping off place
when looking for indicators of
compromise in your network or
vulnerabilities in your network traffic
where possible malcolm
correlates common fields from across
different protocols
to allow you to view one device or
application's network traffic in the
context of the other traffic occurring
around it
for example multiple failed http
authentication attempts followed by a
successful authenticated http post
operation
followed by i don't know
sequential reads and writes
to a file server could indicate that a
foothold was obtained in an http server
that allowed the adversary to pivot to
another service on the network
so a good example of this is the actions
and results dashboards
in which actions
a lot of network protocols have the
concept of of action
and
you know a cause and effect action and
result
um request and response whatever you
want to call it and so actions are
things such as a file was written a
logon was attempted a web page was
requested and then the results would be
success access denied page not found
across all the protocols that that i can
figure out what those actions and
results are i normally normalize those
to the same fields so that in for
example the actions in results dashboard
you can see
actions and results across all these
different protocols together
in addition to the overview dashboards
malcolm provides dozens of dashboards
tailored specifically to application
protocols
including protocols commonly found in
industrial control system networks as
well as those found in more traditional
i.t
kinds of networks
the discover view cabana's discover view
enables you to view events on a
record-by-record basis
similar to a session record in archime
which we'll discuss in a moment
in other words the discover view allows
you to look at an individual line from a
z clock an individual record from a z
clock
the data table in the discovery view can
be customized to display only the fields
that are relevant to the traffic that
you're interested in so for example if i
wanted to put together a play-by-play of
an http session
rather than looking at this big
you know
json log that contains all the fields
that may not be relevant to my traffic
right now
i could go to the discover page filter
on seek.log type for http
sort by time and then include source ip
user agent referrer deskt
and then the http host uri and status
message right
and and by doing that i've got
a more um
focused view of http traffic that i can
then go through and see this happen and
then that and then this
and then once i've got a configuration
that i like for a particular kind of
traffic i can store i can save that
search as you know http traffic analysis
or whatever i want to call it and then
return to it for further investigation
in the future
the visualizations page allows you to
view and manage visualization components
which are are like graphical building
blocks to be used in dashboards
cabana includes lots of different kinds
of charts tables maps for displaying
your data
well cabana is great for at a glance
views and for creating custom
visualizations
archime which until recently was known
as molok provides another interface for
examining
network traffic that may be better
suited to
in-depth analysis and network forensics
earlier when we talked about the malcolm
p-cap processing pipeline we mentioned
that that pcap data um got sent two
directions right it got sent to zeek
and uh and then it also got set down to
mullet capture which is our committee's
program for ingesting that data
while malcolm's cabana dashboards are
focused on the zeke logs and the
archimedes sessions won't necessarily be
reflected there
malcolm's instance of archimede can be
used to view both z-clogs and arcane
sessions together in the same interface
another really great strength of archime
and and i mean arguably its
kind of initial reason for existing is
as a full p cap
a full p cap solution right its ability
to tie the session metadata back to the
original packets bytes
the original packet payload
which allows you to view and search and
export the data
deeper in the pcapp that might not be
referenced in the metadata so archimede
what it really allows you to do which i
think is
so so incredibly powerful is efficiently
deal with very large pcap file sets
and still have access to the underlying
payload data
something that wireshark struggles to do
so similar to
kibana
we want to learn how to
effectively build filters in archime
um to to narrow in on the data that's of
interest to us
um so we've got at the at the top of the
archime interface you've got controls
for specifying your time filter to
define your time search frame
your search time frame and
just like in kibana that's very
important make sure you know what the
time frame of the data that you're
looking at is
there's a
little globe icon that can be clicked to
expand a map filter that allows you to
restrict results to a geo location which
may be of interest if you're looking at
data that that is going out to the
internet
there's a query bar where you can
specify queries in archime syntax
and then
to the right of the search button
there's this eyeball
this the views button and that allows
you to overlay
uh
previously specified filters onto
the current sessions filters and so for
convenience malcolm provides several uh
archemy pre-configured views including
some that involve the zeke.log type
field so that in
archime as we're viewing both archimedes
sessions and z-clogs you've got a quick
way to say hey right now i only want to
look at arcane sessions that are tied to
pcap files or i want to look at zeek
logs that
were generated from pcap files but don't
leak directly back to the payload like
the arcane sessions do so
you can see those here the
some of the views that malcolm has pre
pre-configured for us archimedes
sessions tab provides a low-level
details of the sessions being
investigated
in a way similar to how kibana's
discovery interface does it
and in the sessions view you will see
archime sessions that are created from
pcap files and then that archime session
log is written to the elasticsearch
database
and you'll see zeke logs mapped to that
same archime session database schema and
you'll see those together in the same in
the same pane of glass
it should be noted that you can
distinguish between the two
by the value in or the absence of a
value in the zeek log type column in the
sessions
in the sessions table
similar to how we did with the
discussion to the discover table
earlier in our discussion you can also
customize the set of fields present in
the sessions table
and then you know save or later recall
those configurations of fields that
you're interested in to exist in that
table
as mentioned arkhami's ability to
tie a session record back to its
original packet is is one of its
greatest strengths so details for an
individual session or log can be
expanded by clicking on that plus icon
on the left hand side of the
on the left hand side of each row in the
sessions table
for archime sessions records
an additional packet section will be
visible underneath the metadata section
when the details of this section uh
when the details of a session
of this type in other words an archime
session are expanded archime will will
reach out to where the pcap is stored
and um extract the payload for that
session for display here
various controls can be used to adjust
how the packet is displayed
personally i like to enable natural
decoding and click
show images and files um and and that
produces you know visually appealing
results to me for for when i'm looking
at payload data
but there's lots of options there um
other options also become invisible when
you have uh pcapp data available when
you have that pcap session data to be
extracted for the payload and that
includes downloading the pcap itself
downloading and generating carving out
if you will a pcap for that particular
session
carving out and downloading or viewing
images and files applying decoding
filters
and examining payloads in cyberchef
all that can be done from this
packet payload section
back up at the top of the interface um
if you click the down arrow to the far
right of the search bar you'll see some
new actions presented there including
pcap export
when a full pcapp sessions are displayed
the pcap export feature allows you to
generate a new pcap file from the
matching archime sessions including
controls for which sessions are included
in other words open items only the ones
that i actually got expanded right now
visible items which is everything that
i'm seeing on this page right now or all
matching items everything matching my
current search filters
and then whether or not to include link
segments
once you've defined how you want to do
that
what your filters are and then which
sessions you want to include click the
export pcap button to generate the pcap
and after which you'll presented with a
browser download dialog to save or open
that file in y shark or whatever it
happens to be
note that dispen
depending on the scope of your filters
this could take a long time right to
generate that pcap file or it might even
time out so it's it's a good practice to
look at the number of matching sessions
there on your arcame sessions interface
before you go exporting a pcap and if
it's like hey you know this is
one billion sessions i may want to apply
further filters before generating that
pcap file
to further narrow my um
narrow my search
um note here as well and and this is
kind of a known issue that that i hope
to be able to figure out a way to
resolve but um
you will probably get an error if you
try to do an export pcap without
applying the pcap files view with that
little eyeball icon to the sessions
first
there's there's
a bug in what i'm doing i guess or or
something that when
it tries to export a pcap file from
logs that don't have pcap associated
with it
you might get an error so uh if you'll
if you'll apply that pcap files view
first you'll make sure that that that
error is avoided
um moving on from the sessions interface
uh we're gonna go to archimedes spy view
spy stands for session profile
information and the spy view provides a
quick and easy to use interface for
exploring session or log metrics
basically what the spy view page does is
list categories for general session
metrics like protocol source and
nation ip addresses source and
destination ports
and also like all of the various
different network protocols that are
understood by archimean zeke so whether
it's http or snmp or bacnet or whatever
these categories can be expanded and the
top n values displayed for whatever
field of interest
exists there including that field's
values cardinality
in other words it's a it's a good like
top talkers top n whatever um
display of of any field of interest for
you
in uh in malcolm's logs and between
arcme and zeke
data sources malcolm's list of um
of fields that are available for you to
check out here in the spy view is is
over 1300
different fields across all these
different kinds of network traffic
so click on the plus icon to the right
of a category to expand it
and the values for specific fields are
displayed by by clicking that fields
description in the fields list
underneath the category name
that list of fields can be filtered by
typing part of the field name in the
search for fields dialogs to display
uh in this category text input
um
the the load all and unload all buttons
can be used to to just like
bring forward everything that it knows
about that protocol at once um but you
may want to be careful with this as it
is going to run a lot of queries and and
it might take a while so
you know if if you know you actually
want to see everything yeah go ahead and
smash that load all button but um
it it might you know end up giving you
more data than you really want depending
on on what your interests are
uh once displayed a field's name or one
of its values can be clicked on to
provide further actions for filtering or
displaying that field or its values so
of a particular interest might be the
open field name
spygraph option when clicking on a
field's name or pivoting to another
sessions tab with that field filtered
that will open a new tab with the
spygraph populated or the sessions
populated with that filter already
applied
note that because the spy view group
page
can potentially run many queries spy
view limits the search domain to seven
days or in other words seven indices as
each index represents one day's worth of
data in malcolm so when using spy view
basically that means you need to limit
your search time frame to less than or
equal to seven days before you flip over
this tab or or it'll
complain at you
for doing that
spy graph is is another really cool way
to visualize
the top n field values of a particular
um field
both chronologically and geographically
um so spygraph session profile
information graph visualizes the
occurrence of of whatever fields values
over time
and and for me that's particularly
useful because it helps me identify
trends in a particular type of
communication
over time
when i'm looking at uh
just kind of the conglomeration of all
of the traffic over the date histogram
they're up at the very top
it's hard it's hard to pick out patterns
for a particular protocol or a
particular
ip address or user agent
so
traffic for example using protocol as an
example traffic using a particular
protocol when seen sparsely at regular
intervals on that protocol's date
histogram in the spygraph could indicate
you know a connection check or polling
or beaconing
and
having it split out by protocol by value
like that
is very useful
but it doesn't have to be protocol it
can be any of those 1300 fields that
malcolm knows about you can set as kind
of that pivot value for the spy graph to
to split out those top n values and show
you chronologically and geographically
where those took place
controls can be found underneath the
time bounding controls for selecting
that field of interest
for the number of elements that you want
to be displayed for the sort order and
for if you want to periodically refresh
that data view or not
uh one of my
favorite ones to play around with is the
connections view um but also just from
being cool it is very useful the
connections view presents network
communications via a force directed
graph so it makes it easy to visualize
logical connections between
these logical relationships between
network hosts or between
uh subnets or between
you know again any fields any of these
1300 different data points that we have
you can visualize kind of how the
traffic goes between source and
destination
based on that field values
controls are available for specifying
the query size
so
this again can run a lot of queries and
so
by default like with the query size set
to to small which is like 100 i think is
the default
um
you
it'll run faster but you actually may
not be getting all of your data right
so
what i like to do is set up my other
filters and get everything kind of the
way i want it with my my query size set
to small and then once i've got it all
set the way i want it with my filters
and my views and like the source and
destination nodes set to the fields that
i'm interested in then i will increase
the query size
uh to like the max value so i can you
know see everything
that i'm interested in there
but that will take longer to execute
right
so you can select the query size you can
select which fields to use as the source
and destination for node values
you can set a minimum connection
threshold
the method for determining the weight or
the thickness of that line between the
nodes and and the size of the nodes
themselves
as is the case with pretty much every
other visualization in archime that
graph is interactive so by clicking on a
node or clicking a link between two
nodes you can you can apply or modify
filters
you can
reposition the notes themselves by
dragging and dropping them
um
a note's color indicates whether it's
communicated as a
source or a destination an originator
responder or or both
so while the default is uh source and
destination ip for those fields
um the connections view is able to use
any combination of any of the fields
that archime knows about so some that i
have found interesting or useful before
selecting source oui and desto ui to
view which
devices from which hardware
manufacturers are speaking to each other
source ip to protocol can be
a good way of visualizing
what hosts are communicating to what
servers out there based on the services
that they provide
maybe originating network segment and
responding network segment if we
populated our network segments at the
very very beginning before we even
created our pcap file in that define
network host and subnet name
interface
maybe originating
goip city and responding city could
be interesting to see where my source
and destination traffic is is going out
to on the internet so any of the
combination of these or other fields can
be used to specify that source node and
destination node in the connections view
another cool recent addition to this
feature
and one that actually we developed here
at the inl and then contributed back
upstream to archime
was the ability to specify a baseline
time frame in the connections view and
then visualize use that baseline to
visualize changes to a network over time
in other words you know new hosts or
protocols that appear to my network this
week that didn't exist last week
this feature is mainly useful if you
have prior long-term packet captures
available in order to establish that
baseline right in order to define what
my previous time frame is versus what my
current time frame is so that you can do
that comparison
another really cool feature of archime
uh that allows you to um
to get access to the payload data and
actually search the packets themselves
versus just the session metadata is the
hunt feature
you can kind of think of this as like a
a pcap grep um the the search string
that you specify for the hunt filter can
be specified with ascii
uh with or without case sensitivity uh
hex codes regular expressions and
basically you you create this
packet search job on the hunt page with
the filters and the other parameters
that you're interested in to limit that
search um
scope
to make that packet search go a little
bit faster um and then once that hunt
job is complete it runs in the
background but once it's complete it
tags matching sessions with that hunt id
and you can go view the sessions or the
matching uh payloads in the sessions
view
um
so
note that whatever filters you specify
in the
search bar when that
hunt bar when that hunt job is created
will also apply to the hunt job as well
so if i'm over on the sessions view and
i've got a filter in the query bar and i
switch over to the hunt view those those
filters
limit the search of packets the search
of the scope of the packets the scope of
the search of the packets that i'm
looking at in that hunt job
and and so pay close attention to that
um there's a little little like
information icon that says creating a
new packet search job will search the
packets of
1 million sessions or whatever that
number is just be aware of what that
number is because um your
your search your hunts execution time
will be directly related to how many
packets it needs to go out and look in
note that the hunt view is only
available for sessions created from full
packet capture data not z-clogs in other
words archimedes sessions only not
z-clogs
so it's a good idea again here to apply
the to click that eyeball icon click the
view eyeball and select the pcap files
view to exclude z clogs from your
candidate sessions um prior to using the
hunt feature
a couple of notes
on
correlating data sources between the
archime and
zeek
data sources
note that
because these are different tools
developed by different organizations for
different purposes
the search syntax between archime and
cabana is different right malcolm is
utilizing both of these open source
tools together but um
you know in some cases the
in in all cases really the search syntax
will be a little bit different between
archiving and kibana and in some cases
the field names themselves will be a
little different
so um
you know refer to that documentation
refer to that table that we showed you
earlier uh to to compare like how do i
write a query that
um searches an ip address based on its
presence in a subnet uh you know or for
sessions that that include that subnet
or for how do i do a regular expression
or whatever
because it won't necessarily be the same
in our community versus cabana
um
the as i mentioned archimede uses its
own field names in its user interface
for example in archime you would search
protocols equals equals http
but in kibana the equivalent search
would be protocol colon http
so going to archimese help
which is the click on the owl icon in
the upper left hand corner and then go
down to the fields section which is at
the bottom of that help page and click
uh database display display database
fields can help you map what archimedes
field names are to uh what the
underlying database field names are that
the um that the z clogs might use
as much as possible during ingestion we
do try to map those arcane fields to
sorry the zeke logs to their
corresponding arcane fields um
but uh but it might help to know like if
i'm jumping from arkham over to kibana
and i want to know what fields it was
actually searching when i was in archime
you may need to do some manual
mental mapping of the arcane field names
to the underlying database field names
also you know despite considerable
overlap especially for common protocols
there are differences in protocol parser
support between zika and archimede
notably malcolm's configuration of zeek
parses a lot more ics protocols than
than archemy does because that's one of
the main focuses of this project
so we've looked at these two different
interfaces to uh to analyze that same
underlying um
the session metadata from our network
traffic
we've taken this one p cap and archime
has generated its session records and
zeke has generated its logs
and um
how can we correlate the two how can we
how can we bring them together and and
see like this get the strengths of both
tools right get the information about
the
protocols and stuff that our community
maybe doesn't have support for
from our ziklogs but then at the same
time be able to see the archimedes
sessions
that correspond to those
events of interest or whatever they are
from the z clogs so that we can get down
if we want to and see the payload and
and do analysis uh kind of at that level
um so as i mentioned just in in the
previous topic like
one of the things that
that malcolm does to try to facilitate
this is it maps
wherever possible
zeke
fields to corresponding archime fields
in the database schema
but then for any protocols or fields
where um
where archimede doesn't already have
native support
malcolm creates a native uh
database uh a native data source type i
guess for that kind of zika log
for all these other z clock values for
which there's not currently
support or an equivalent in archime
the fields section of archimese help
which i referenced a few minutes ago can
provide a list of all of those known
fields across both the arcame and the
zeke data sources
so in this way when full packet capture
is an option
analysis of pcap files can be enhanced
by the additional
information zeke provides
why i say when pcap
full packet capture is an option um you
know there may be cases where for
whatever reason whether it's size
constraints or um
uh you know
sensitivity of the information like you
may be able to to capture z-clogs just
see clogs and actually not store the
full pcap so
in that case malcolm can still be used
in both the archimede and the cabana
interfaces but when you have the full
packet capture available to you um
you you are able to
you know
enhance that full capture beyond just
what the arcane sessions giving you with
with the z clogs as well versus the z
clocks just being alone
the value
the values of the records that are
created from zeke logs can be expanded
and viewed
like any native arcane session by
clicking that plus icon to the left of
the record in the sessions view just as
we showed you when we were talking about
the sessions view
however note that when when you deal
with those zeke logs
uh the zeek records the full payload
that section on the packet payload
doesn't exist right because the pack
contents aren't available
so the buttons that deal with viewing
and exporting pcap information
like don't behave the same they don't do
anything basically as they would for
records with pcap files
other than that though zeke records and
their values are usable in malcolm just
like their native archimedes session
counterparts
a few fields of particular mention that
help limit returned results to those
ziklogs
and archimedes session records generated
from the name from the same network
connection
are community id and zeke's connection
uid we've mentioned both of those
earlier
and we can talk about how
let's talk about how we can use those to
to get kind of that full picture right
so so the example that we're talking
about here is something interesting
happened on my network
i want to see in one list
everything i know about it i want to see
all the z-clogs and i want to see the
arcane session or archimedes sessions
that corresponded it to it
there's kind of a hard way and an easy
way to um
to do that the hard way would be
you know find the logs that i'm
interested in or whatever in kibana
or or however i want to do it and then
like say okay
i've got this source ip and that desk ip
and this port and it happened at this
time and just kind of try to like hand
craft your filters to
to include everything you want but
nothing you don't want
and that's difficult to do but what we
can do is um we can use this z
connection
uid zeek.uid and that community id and
we can use that to kind of build a query
filter that includes everything that has
to do with that session across both zeek
logs and arcane sessions
community id is a specification for
standard flow hashing published by
corelight and the intent is to make it
easier to pivot from one data set like
archime sessions to another data set
like z con.log entries
in malcolm both archime and zeek
populate that community id value which
makes it possible to filter for a
specific network connection
and see both data sources results for
that connection
that zeke.uid is also mapped to another
archimede database field called uh
root id so you can use zeek.uid or root
id interchangeably that root id field is
used by archime to link session records
together when a particular session has
too many packets to be represented by a
single session when normalizing z clogs
to archime schema malcolm kind of
piggybacks or hijacks on root id and
stores that
connection uid to cross-reference uh
entries across z-clock types
so um
that that's interchangeable zeke.uid or
root id
so
the the cool pattern that i want to kind
of get across here in this example is
by filtering on the community id
ored with the zqid so community id
equals equals some horrible long string
or pipe pipe
root id equals equals some different
horrible long string that represents the
zqid
that will that will cast this tent which
will include both the archimedes
sessions and the z clogs
that are generated by this particular
network connection and you can see them
together in one place
one item that was
mentioned earlier but we didn't really
get into the details of how it works is
the
malcolm's ability to analyze files that
are carved from network traffic
as i did mention earlier as i referenced
this feature zeke can carve files from a
variety of protocols
in observed network traffic um
and then those files can actually be
extracted and stored temporarily locally
by malcolm for analysis
malcolm leverages that feature to submit
these carved files to a number of file
scanning tools
clam ev for example which is an open
source antivirus engine
can be used to scan for known malware
signatures
yara
the pattern matching swiss army knife
scans the files using a curated list of
security related signatures or your own
custom ers signatures that you can write
kappa which is a portable executable
capabilities analyzer
and virustotal an online database of
file hashes now to use virustool you do
have to specify your api token and you
would need to have an internet
connection for those um to be
submitted for those hashes to be looked
up so malcolm can be
configured to
there's a couple different other
configurations with the file scanning
behavior that you can set number one you
can set which kinds of files you
extract and scan to begin with
meaning
do i want to scan all files do i want to
scan
files
that are just of mime types that may be
of particular interest when it comes to
a security standpoint right things like
zip files or executable files or pdf
files that might be vectors for for
common known attacks
and then additionally to which file in
addition to which files we scan
we can specify
which files we want to preserve if any
in other words do i want to preserve all
files and and then to get stored in this
directory locally on the malcolm
instance
or do i want to only preserve files that
get hits from the scanning engines
that might be flagged as suspicious
or do i not want to preserve files at
all and then those files if they are
preserved can be downloaded from that
directory where they were preserved and
then you can do whatever further
examination you want on them whether
that's submitting them to other file
scanners or reverse engineering them
with with gidra or or ida pro or
whatever your
tool of choice is there
as the
files are scanned
if zeek file carving is enabled and
those scanners are turned on
questionable files will be written into
the signatures log and reported on the
signatures dashboard in kibana
the signatures dashboard will break it
down by the scanning engine and then
also the name of the signature
or rule that triggered on that file
the z connection uid zeek.uid and file
uid zeek.fuid
fields in these logs in the signatures
log can be used to cross reference to
other visualizations to provide the
context for how that file was
transferred so in other words you know
finding on the signatures page a
reference to a zeke.fund for a triggered
signature filtering on that zeke fu id
and then jumping over to the files.log
to see what what is it about this file
that i know how was it transferred how
big was it what other dashboards might
reference that file that sort of thing
just a few search tips before uh
before we close out here um
to effectively search in cabana and
archime um number one always check your
search time frame if you're not seeing
the data you're expecting to see often
it's because the data lies outside of
the window of time that you're searching
number two this effective technique for
investigating is is
zoom in
for example narrow in on a particular
type of file that's transferred or a
particular host
once you find something of interest
pivot to another field so
select the source ip address that was
the source of this file transfer
and then zoom back out by removing the
file type filter to see what other
activity that source ip was involved in
that that's a very effective way when
dealing with a large data set is is you
find some value that's of interest to
you
zoom in on that value see the contextual
the other values in context with that
and then zoom back out by filtering on
these new values and removing the
original filters
remember that most elements in kibana
and archime are interactive and can be
configured to work with any of the more
than 1300 data fields that malcolm knows
about so
generally in archimede kibana if you can
click on it you can you can create a
filter or you can pivot to some other
view and
and
you don't have to go
find these things manually uh you know
don't don't
save yourself some work by by learning
to create filters from the graphical
interface uh rather than just having to
go remember and oh what was my query
string that i was doing before and then
and then get to it that way
um
learn how to filter on on common fields
like zeek.log type
the the source ip desktop source port
desport protocol field
action and result
oui source oil ui and dust oui
these fields that get normalized across
all of the different log types that we
can do uh with zeke um you know there's
there's you're not gonna memorize all
thirteen hundreds of the fields that we
have uh
that we could populate in malcolm but
you know as you start to become familiar
with those most common ones
learn to build filters around those
fields and you'll become more effective
in your searching
finally don't forget about that tags
field
using pre-populated tags like the ones
that we do during enrichment for the
private and public ip space
the tags that you that got generated
when you uploaded your data
the tags that got created during the
segment mapping stage of enrichment
all of these tags including tags that
are populated automatically based on
pcapp file name
learn to search on those tags apply
filters on the tags field
and you can narrow your you can you know
easily narrow to cross segment traffic
with tags equals cross segment
and it's much harder to do the same
thing with source ipn
you know
192.168.0.0 slash whatever and
desktopn10.whatever.whatever like yeah
you could do it that way but it's going
to be a lot easier just to use the tag
that's already populated for that field
i hope this was a good kind of malcolm
101 course for really starting to get
down and get our hands dirty on
with pcap analysis
i enjoy
maintaining this project and and showing
people how to use the tool
um if you've got feedback for me
reach out on on github
uh reach out via if you want to in the
youtube um
comments or uh or get in touch with me
any other way you can figure out
and and let me know what you think
we're interested to to know how the
community is using the tool and and ways
that we could improve it
thank you for your time and happy
hunting