hi everyone
i'm mike and uh today i wanted to speak
with you
about velociraptor i wanted to go
through a few examples of how we can use
velociraptor in a in a real dfir case
you must have heard of velociraptor
velociraptor is a unique dfir tool
that gives you the user the power
uh to be very flexible about the way
that we collect
uh artifacts or information from
endpoints
and the thing that makes it really cool
is that it has this very powerful
velociraptor query language
and i'm going to show you today how we
can use that to
do some advanced hunting so it's a very
short talk so we're going to go
and skip a lot of the information but
there's going to be some references at
the end so you can have a look at some
more information about this
so let's just take a quick look at the
overview what does velociraptor look
like
so it's an agent-based system we have an
agent running on different assets
and we have the velociraptor server
usually it's deployed in the cloud
and the um admin ui which is what we're
going to be using today
we can use that to um to task and
control the the server
the nice thing about it is it has
persistent communication
to the to the endpoint so we don't
really have to pull or anything we can
just go ahead and
and query the endpoints interactively in
seconds so i'm just going to go through
a quick tour of the the um the
velociraptor ui because i'm not
going to show you guys how to install it
because we have a lot of references
about installing it
uh already um but i just wanted to just
point out that it's a very fast very
efficient
scalable system and typically uh we're
looking at about 10
000 endpoints on one server so it is
actually pretty efficient we can do some
very very fast hunting with that
so let me just show you a quick
velociraptor gui tour
um let me switch to the ui so this is
the velociraptor ui once you have it
installed
and the first thing that you can do is
you can search for different machines
uh let me just show you the sidebar here
uh the home page is the dashboard we'll
just have a look at the dashboard here
and you can see that i have this server
running for a while
um and it has this is the memory usage
of the server just a bit of monitoring
and this server has about 2 000 clients
connected to it we're going to do some
hunting
in in just a minute uh what i will do is
first i'll show all the
all the machines there's a whole bunch
of them and it just goes on for ages
uh but we can actually search for
specific machines by label so we can
label a machine
and we just we just kind of highlight it
here and add a label
uh and that allows us to kind of group
machines into uh
into groups so we can hunt hunt uh in a
more targeted way
i have a machine here which is my
machine here which i've labeled uh
as mike the labels mic
so let me just find it
come on
let's label it enable mic
okay um i'll just find i'll just
show you on another machine let me just
quickly have a look here the dashboard
we've seen that
yeah and we can investigate individual
clients
interactively so let's just pick a
client and so
this is just some information about this
client
if we have a look at the vfs we can
actually go and interact with files on
the machine so we can you know go
through
and at the moment uh this is just
showing us what
information the server has on this on
this machine so we can just
uh refresh that by
clicking that that just syncs that
directory listing so we can see what's
what's on there
we can also do a recursive directory
listing and you know see
a whole bunch of files uh and on that
machine
and you know we can we can go in and
you know download individual files or
whatever right from from that machine
so when we look at a file let's say
end user that this one
we can simply click collect from client
and then it just collects that
that file from the from the machine and
then uh and then we can
you know look at it uh let me just stop
that
okay so i'm just gonna go through that
part very quickly because i wanted to
show you guys
uh a cl example of what we actually use
uh oh i just wanted to show you quickly
the uh
interactive shell feature that we can
actually go through
and we can get a shell on that machine
at
any time so for instance in this case if
i wanted to know what
uh what are the local users on that
machine
you know i have a little powershell
little powershell snippet here
i can simply paste that here and
and collect that and that that just
basically runs the file shell on the
machine and
we get back the responses so you know in
this case this particular
powershell just tells us what the local
admin users are so so we've got this um
yeah okay so let me um
go through and so we've seen how we can
fetch files
and interact with the machine but what's
really cool about velociraptor is
because we have that query language
that we can use to uh to really
customize the way that we are actually
interacting with the machine and
querying things
i just wanted to show you an example of
uh what we call a forensic artifact or a
velociraptor artifact
uh and we we use that term a lot and
you'll see that a lot in the
documentation
of the artifact the term artifact
so when you look here in the ui we have
our artifact viewer and you can just
click on any of these
artifacts randomly you'll see that these
artifacts are essentially just queries
that are packaged up inside a yaml file
i'm not going to go really into details
about the query language itself in this
talk
but you just need to know that these
artifacts are just queries
that are named so they have a name and
so we can actually just
use those queries and just ask the
endpoint about them
at any time so just to give you an
example of
how useful it is having these uh
artifacts in the yaml
form allows us to interchange them and
share them with the community
so i'm just going to go through an
example as a really quick one and this
is just something that i just put
together in the last few days because
it's actually like
a current zero day you know that at the
time of recording at least
um it is the serious sam or hive
nightmare example
so let me just go through that uh and
you can see that
there is a a uh cve for it and there's
an advisory out
and you know we can have a look at you
know the different
resources there are many resources that
explain this vulnerability but
essentially
what this vulnerability is if you read
the details
is that there is just a weak permission
on the sam file
so if you look at it the built-in users
group
has read permission on the sander
on the directory where the sam is
contained and that allows you to
allows low privileged users to just read
the sam crack the hashes and
escalate to domain admin from there so
it is actually quite a serious
vulnerability
and so you might have you know a whole
bunch of machines and you can say oh
well how do i
how do i fix that well you can see the
the advisories and microsoft has
released an advisory about it
and we need to check the acls but how do
i do that to 2000 machines
how do i check that you know 2000
machines are patched or whatever
so this is where a query language really
shines right
i can the idea is that i could write a
query
right now and run it on all my 2000
machines
and then see the results uh in seconds
right so
um so let's let's have a look at some at
our
public artifact reference which is a
public place where we can share
these kind of uh artifacts these queries
so we this is our website our
documentation website and we have a
thing called the artifact exchange
the artifact exchange is a place where
we can change
uh information about current threats
and people can write different queries
different artifacts
that we can share with the community so
you most of the time you don't really
need to be able to write your own vql
you can just uh have a look at the
artifact exchange to see whether there's
something similar
you see we've already got something here
but the one that i just wanted to show
you today was this
uh access control list uh artifact which
is
uh allow which basically pauses
uses uh powershell to get the access to
the acls of the um
of of the sam uh um
the sam files right so i'm not going to
go again into too much details here but
all you have to do is really just
find the artifact that you're interested
in and then you just
copy it from here and then we're just
going to go
back to our server into the artifact
viewer and we're just going to paste
that artifact into our server right so
this is
very simple we just click the plus
button here
highlight delete you know paste
and and that just pastes the artifact
from the
artifact exchange so so you can just
directly use it i mean if you
you can pretty much see that that's the
first part that's the powershell
script a very small partial thing and
then we wrap it
in vql and then we do some filtering
there with uh with extra
extra processing right so it's a pretty
straightforward
approach but now when we save it
then you'll see that it actually we have
a customized artifact
it's part of the ui part of vel's raptor
and we can just use it
in any context that any of the other
artifacts can be used in
so uh let's go back to our to our
machine
let's see if we can find it again yep so
this is my
mic label right so this is a machine
here and
now i'm just going to look at that one
machine here
um over here on the top i've got the
name the host name of the machine and it
says connect it so it's now connected to
the machine so we don't really
pull we are directly connected to it so
when we task it
we can immediately um immediately uh
get information from it so i'm just
going to go to the collected artifacts
and velociraptor just deals with
artifacts that's all it knows about
so this is a list of the artifacts that
are collected
when when we first start we collect this
information general information artifact
but what i'm going to do now is i'm
going to click this plus button here
and we're going to collect that new
custom artifact that we've created
which is the access control list so i'm
just going to select that
i can search for it first and it again
shows me a bit of a description of it
reminds me about it
but you can see that there are a couple
of parameters here that i can use to
customize this artifact that i've got
from
the artifact exchange so i'm just going
to configure those
and you can see that i can change the
glob so i can you know check for other
files and access controls in this case
i'm interested in the sam
specifically so it's it's going to be in
this directory with an
s star and this is the acl filter
that i'm specifically looking for i'm
only interested in files that have that
uh that permission that filter so
because this is the vulnerable
permissions
so once i do that and configure it then
i can just launch it
and uh and off it goes so this is going
to go to the endpoint to that endpoint
and run this powershell thing
and uh and then it will come back with a
result that would just be all of the
um oh see it's finished and
it's uploaded four rows so just like
velociraptor is a query language
really queries only just return rows let
me click on the results
and we can see here that it's showing me
the sam
this is the owner administrator but
these are the acls on it
and you can see that this is the kind of
problematic acl here so this machine is
vulnerable
to this to this uh hive nightmare
uh zero day so so that's cool and so i
could i could do that and i could
and i'll show you in a minute how you
can do a hunt over thousands of machines
and find out
in which ones uh of your machines are
vulnerable in seconds
but uh now what am i going to do about
this i mean like okay so i know this
machine is actually
so this is just a big of a recap to show
you just the general structure of this
uh vql query and how we adjusted the
parameters and we could see the
uh the vulnerable thing so how do we
what are we going to do about this this
machine is now vulnerable so we actually
really need to remediate it
now i mean of course we can always log
into that machine and use our
remediation steps
and in fact if we look at our references
there are some remediation guides
and particularly we need to run this
command here
but this command uh so you know we can
do that to one or two machines but we
want to really be able to do it
to many machines and this is where the
velociraptor can really help us
because we can just change this vql to
add that extra command here
the extra remediation step and then and
then we can use that
to remediate that machine so let's let's
give that a try
okay so this is the remediation step
and what we're going to do is we're
going to go back to our
artifact here and uh and instead of we
can now customize that so the one that
we copied from the
uh from from the artifact exchange is is
good it just tells us about it
but we want to change it so we're going
to just click the pencil button here to
customize
and and what we're going to call it is
uh you know remediate
for example we change the name so it's
going to be a different artifact but
it's very similar right like essentially
the same thing
except we're going to add that that
thing here um and
what we're going to do is we're just
going to um
uh out let me just check this uh
this format what it was supposed to be
out null okay so
this one basically just runs it without
uh
without any output because we don't want
to mess up the um
the json that we are we are extracting
right so so we're just gonna
as an extra step run this extra
remediation step and then
we're going to look at the permission so
hopefully that will show us the
permissions have been fixed
after we do this so we're going to save
this so now this is the remediation step
and i mean generally we want to be very
careful we're doing remediation because
you know we don't really want to mess up
the uh the system right
but let's let's have a look so uh so
we're going to do a new collection i'm
going to collect this new one
this is the remediation one and we're
just going to launch that so
it's doing the same thing except it's
going to do that extra powershell
snippets
to just clean up the machine and then
hopefully
uh once we once we finish uh we should
see that
uh that uses one uh disappears so um
that's hopefully it worked
let's remediate it sorry
let's check that again
and hopefully that that would have
worked
so this gives us an idea of how we uh oh
it's still there
maybe that remediation is not working
too well
uh
it should run the remediation step maybe
it's
oh yeah that's right sorry
this remediation is not for powershell
so we're just going to do this
windows system32
okay so let's do the remediation one we
can copy it again
and launch it around the same thing but
with the modified one
and this time we should expect to see
that command
worked and yeah it returned zero rows so
it actually worked correctly
and there's no more results so now if we
do that
access control is check again so we just
copy that same artifact collect it again
second time uh and then it should return
zero rows because now we have no uh
vulnerable sam the sam is no longer
vulnerable right so it doesn't have that
acl
on it because we've we've fixed it right
so
again we have to use the correct
powershell version of the
of the of the fix but once we do that we
just edit it through partial and it just
goes off and does it so we can now do a
hunt
to uh to remediate all of these machines
okay so that's that was pretty cool um
so let's
have a look at the second example and
this one is a little bit more involved
and it it basically uses a bit of
research
so in this second example is about
disabling log files
and uh and you might not know
that you can actually download things
in windows using the beats downloader
that's a pretty common
this one's a pretty common uh
persistence that a lot of
a lot of people people use right so
attackers use so here's an example of a
command line and it uses something
called
the bits service right so the bit
service is used to download the windows
updates and
various other updates but essentially
it's like curl basically those often
downloads
stuff from the network now if you have
something like an edr
that watches to see you know oh is is
there a powershell connecting out to the
network and
it could be suspicious so a lot of
attackers use this beats admin
to download their malware because uh
it just bits admin is part of windows
and it just it's usually whitelisted
and signed and so it goes off and a lot
of videos just let it go
because they don't uh monitor what
exactly it's downloading so you can
actually use it to download
anything including you know any any
powershell or whatever you want
as as a a download cradle
so let me just uh try to uh to do this
as an example so we'll just open up a
shell
and what we're going to do is we're just
going to download uh
let me just put it in mic test
okay and if i just run this command then
it's going to go and it's going to
create
a bits job and goes off and downloads
this
google.com into uh
see users like test
dot oops dot ps1 right so that
that is what it went off and downloaded
that now so
this is actually kind of suspicious
thing and a lot of people know about
this so
usually you actually can monitor for
that
using the event viewer so if you look at
the event viewer uh
then often you will see an
event and the event is in windows beats
clients operational right so you see the
event
and this is pretty suspicious a lot of
people have like event forwarding
and they forward that to their theme and
you know and then there's alerts and all
this kind of stuff and it escalates
right because this is actually pretty
suspicious
when you look at uh at this event you
can actually see
who downloaded it oops the username and
you can also see
um in one of the other events where it
was downloaded from
and who created it so you can see this
is the url
that we downloaded from this is what
they downloaded and that would
set off a lot of flags with powershell
is coming from the internet so but
i'm not going to talk to you about this
bits thing because it's well known what
i am going to talk to you about today
is this very interesting uh thing that a
lot of people don't realize is you could
just turn the logs off
so if i right click on this and if i
just
disable the log like so very easy
and if i just clear this log i'm gonna
i'm gonna clear this log
and do it again and show you that
now the log is not logging right because
the log is disabled
so if i had an event uh forwarding type
seem that i'm just gonna you know
forward events from this then i'm just
now completely blind
and all it takes is you just right click
on this thing and you know just disables
the log it's not a big deal right
and so you know if attackers about to do
this they're just you know they can do
that they can disable the login off we
go
so what we would like to know as
defenders is like
has anyone done this has anyone disabled
the log what was the baseline what
should it be
the baseline so first the first step is
figuring out
you know what is going on when i disable
this log
and when i disable this log um what
happens is
that so let's see if i don't know what
it is so usually i would just
uh run proc mode so i have pokemon over
here
and i would start it up and uh
and then just you know
do this disable log business maybe i'll
just re-enable it again now
enable and then i can stop
procmon stop capturing
and i can add a filter i've already
added a filter here just for this talk
the filter is looking for an operation
which is a reg
set value so basically when i change
this in ui something is going to set a
value
somewhere in the registry and i don't
know exactly where
so i'm just going to see a filter for
those
and you can see it very quickly that
when i turned the
event log on or off disabled it then
you'll see a particular key
uh somewhere here uh which
corresponds to that that setting uh
where is it
here this one right so this event
is a windows current version we need vt
channels microsoft bits clients so this
is a typical example of
somehow the attacker is misconfiguring
this system and we want to know
uh what's going on right so we want to
write an artifact here that just detects
this across
across our system so let's uh let's have
a look at this example so this is the
example i have here
um so we did this we looked through uh
we checked about the disabling we did
procmon
we checked this value and in the end
we can write an artifact to check for
that key that
specific registry key and again in this
talk we don't have a lot of time to
to be able to see which uh
to work out how to write this this query
so i'm just going to paste it
so we can actually run this query as it
is
uh and create an artifact from it but
let me just show you
what that query looks like we're just
going to run the query in the ui
so in the ui uh we have a thing called a
notebook which we can create
a new new notebook here test
and in this in this notebook we can
write any vql that we want we can run
different pql so
uh let me create a new cell
pql cell and paste again i'm not going
to show you how to
actually write this in this talk because
it's not enough time but you can see
that
here's the registry key that we are
looking for there's a glob here so
there's a star
so we're looking at all the events uh
all the
logs and then looking to see whether
they're enabled or not so we can run
this query
and what it will do is it will basically
show us which query is enabled and which
query is disabled
right so we can actually write this
artifact using this query
to know which which event log is enabled
which event log is disabled
right so so let's let's uh let's do that
so we'll write an artifact so this time
we're going to create an artifact out of
this query
so just like we did before but this time
we're going to create a new artifact so
we're going to go
to the artifact viewing the artifact
viewer
and we're going to add a new artifact
but remember before we just copied it
from the artifact exchange
but now we're just going to write our
own artifacts so
we just need to give it a name so it's
going to be a windows
event log event log enabled
i don't know we just
give it a name description and so forth
but really the most important thing is
to paste
our query here oops and
because this is yaml we have to kind of
like tab it over a bit and you see as
soon as i do that the the syntax
highlighting helps me
here i have the preconditions so that's
telling me that it's only
this artifact is only running on windows
so i'm just going to delete that
and i don't really want any parameters
here so
i'm just going to just do that so that's
that's my new artifact i just create
that
and there we go we have a new custom
windows enabled event log enables
artifact so now we can simply collect it
from our endpoints right so we can
simply go back to our endpoint
collect that artifact just as we did
before
remember there's no parameters this time
so we just hit launch and off it goes
and that's going to tell us basically
which um
which uh clients are which logs are
enabled or disabled
here we go so the same thing so so we've
gone from
an observation of uh something weeds
going on
how do we hunt for it and then we create
uh
an artifact for it so this is the the
standard word workflow in velociraptor
it's the hunting workflow so we start
off with an idea
we explore the idea in our vql in our
queries once we come up with a query
that we like
then we simply convert it to an artifact
and then we can go hunting this is the
part i wanted to show you right now is
how we can go hunting for this
so what is normal and if you look at
the event log you'll see that a lot of
them are enabled a lot of them disabled
and you don't know which one's supposed
to be enabled that's not enabled which
one's supposed to be disabled that's not
a name
disabled right so you you don't you what
you want to know is whether one of them
has been changed
but you don't know what's normal so if
you don't know what's normal that's
called baselining where we need to guess
get a baseline of what's going on and in
order to do that
what we want to do is we want to collect
this information of all the logs that
are enabled all the logs that are
currently disabled
from all the hosts so we want to do a
hunt a hunt is basically when we go
through
and we collect that same artifact this
one across the entire network
at once so at the moment we've just
collected it from this
one machine here the mic machine but we
want to do it across the entire network
so let's just for the sake of our
argument
let's have a look at this look so this
log here
is enabled right so that's enabled let
me
uh just quickly check my other machine
here
and this is my other machine this is the
mics machine and i'm going to disable
the log just on the
one machine so i'm going to see like
this is going to be the one that stands
out right
and let's go and do a hunt and see which
one
is disabled a lot of machines are
enabled this if this log is enabled by
default but we're just going to disable
it so we're going to go to the hunt
manager this is the hunt manager
and we're just going to create a new
hunt so we're going to add
another artifact we can give it a
description so you know events
logs select the artifact that we want to
collect so it's just this one whether
it's enabled or disabled
and literally just launch it in this
case because there's no parameters it
just goes ahead
when we create the hunt it's paused so
we're just going to launch
start it run it and you'll see that as
soon as that happens
the endpoints are starting to get
scheduled
and they're starting to that number will
increase as they're getting scheduled
each one is going to go off and check
its event logs and send back the results
and then and then it will be finished
right so it's going to be scheduled and
then finished
and and that goes off and this is
basically how we go off and hunt through
all our machines right so once this is
happening
this you know we don't really even need
to wait for it it's going to be
uh we have 2000 machines on this on this
uh
system so it's going to take a couple of
minutes but we can immediately start
uh and analyze the results so this is
the
processing step in the notebook and the
notebook is
where we can write different vql and we
can analyze the results so you see for
example this is
the results that came back from this
machine 87
and you know it has some zeros and ones
but we don't really know which one is
um which one is valid or not so we want
to basically just do a group by we want
to do a stacking
we want to stack all of our results and
then see which ones are different
so um so what we really want to do is we
want to do a group by
uh and then we want to we want to count
right count as count
and then we have the log all the
different log
and the value right um and
the hostname uh
that's the hostname here and then what
we want to do is we want to group by the
combination
of log and value right so
group by basically counts together all
the unique ones of the same value
uh combination so for example this bit
client operational is one
on one machine and then it's zero on
another machine that's a different group
so we wanna just do that right so um so
then when we do that so
uh it's gonna go off and and essentially
just count them
and this way we will be able to see
which ones are common
and which ones are not uh not common
because they'll sort of stand out
this is a pretty classic hunting
technique called you know
stacking where we can just count just
counting different things and we can see
that
some machines are a bit different to uh
other machines
it's going to take a couple of uh
seconds while it's doing that let me
just
jump onto the presentation to just see
what
uh we're doing here so again what we've
done is we we've done a stacking
exercise across all the groups
and we counted all of them oh we should
have also ordered by a little bit
but maybe we'll find when we order by we
can see the counts uh
you know in order so we can see the ones
that are a bit different
um so that's going to take a couple of
seconds while it's
collecting the data from the endpoints
but typically this is what you see
you'll see that that one machine that
has
it disabled so the value is zero it
stands out right
and then the other ones are all the same
so that's more or less the baseline
and then that one is the one that stands
out
all right um so uh
we only have a couple of extra seconds
so i'm not going to show you how to do
this but i'm just going to tell you
uh how we can turn that artifact into
a detection rule and uh this is a little
bit
uh of a more advanced use case
i would say uh and in vql you can
actually create event queries so you can
actually write monitoring rules
that use the query to detect when
something happens
in the future and this is kind of what
it looks like in
on the end point we have the query
running and
it basically ends up sending partial
results as as
as as it happens i'm just going to skip
to the
architecture so we can see that event
queries are running on the client
and then they're getting buffered and
they're sent off
to the server when the when the client
is back online so we can write a query
that simply uh you know detects whenever
this log file is changing
and this is an example uh of this is
an advanced over the typical you know
ultra loop
in in a normal edr type sim system
you you have basically a whole bunch of
events that are going into the scene
and then we have escalation and then it
goes back
through and and then the user and
operator actually goes back to collect
extra information
but we can actually do that on the on
the endpoint by simply adding
a query an event query that runs through
it so to give you an example in this
case
we have i'm just going to skip through
because we are out of time
we can add this particular event
artifact to
to velociraptor that will simply check
uh whether
the event logs are enabled or disabled
periodically
and when they become disabled uh then
there will be a difference
and the and the event will and the query
will basically report a row that would
be different
so an event log has been disabled um
so i think we're out of time to actually
demonstrate that particular
um uh one so we're just going to
uh just show you a screenshot when that
happens
um basically once we install that
artifact here uh
then we will see whenever someone added
or removed
a uh an entry from the event logs
uh from from the registry key so
whenever the registry key changes we
will see
on the server and event raised and so
that's how we can turn vql into a
monitoring type system
so we didn't cover a lot of stuff it was
really quick uh
a really quick talk so i do encourage
you to go and have a look at all the
resources that velociraptor has
so there's many many things that we
didn't cover
and you know we only just introduced it
so hopefully this will give you a bit of
a taste to see how you can scale up
dfir and be able to uh to uh
collect information from thousands of
machines in seconds uh and then in the
last
slide here i have uh a whole bunch of
references here
but really docs.velociraptor.app
is a website it's uh if you go there you
know there's
lots of resources here we've got
training packages announcement uh
documentation
and the artifact exchange you could use
that to
submit to it and so on it's an open
source project
found on github you can visit us on
github file issues
and also uh we are always on discord so
you know help help out any questions uh
or anything like that
so uh thanks for thanks for your time um
and uh and and
have a good conference thank you