hello everyone i'd like to welcome you
to my talk today using passive dns for
gathering business intelligence or how
to snoop on what your competitors are up
to
so i'm going to start with a quick
introduction to the talk
in this talk we will look at passive dns
what it is and a little history
some of the services available to query
passive dns databases
how it can be used for gathering
business intelligence
and finally how you can mitigate
business intelligence leakage
so a little bit about me
my name's andy dennis i'm director of
security at modus create a dc area
consultancy we work across multiple
fields software engineering cloud
architecture cyber security over the
past 10 years we've probably worked on
it if it's an area of technology you can
learn more about us at
www.moduscreate.com
20 years working across software
development architecture management
devops cloud and cyber security
and in my spare time outside of work i'm
an organizer with b-sides connecticut
you can visit our website at bsidect.org
we're hoping to hold the conference this
november
hopefully a hybrid event um but
depending on the covert situation where
you have to revert to a virtual event
only this year but if you're interested
in doing something in november then
please check our website out
i'm also the author of several books on
iot and a raspberry pi and i recently
co-authored a book uh on docker called
docker for developers with two my
co-workers uh richard billington mcguire
and mike schwartz so please check that
out if you're interested in the
subjective containers
so the genesis of this talk how did this
come about
really the question is how do our
customers prevent snoopers gathering
business intelligence
this question was a byproduct of
conversations we had back in january
2020 while poking around in risk iq
while we're drinking some coffee and
then a brainstorming session over beers
later in the evening we wanted to
understand based upon the data we saw on
risk iq how do we avoid leaking
sensitive information about our company
and secondly how do we plug these gaps
is it policy process technology a
combination of them
we're also interested in what's the best
approach for helping customers keep new
products under wraps before launch how
do they prevent competitors
from poking around in tools like risk iq
and finding out sensitive information
about new products they're developing or
perhaps customers they're working with
and finally
we're curious
what are our competitors doing and if we
can see what they're doing
can they come and see what we're doing
of course the answer that is yes if
we've left information available which
is picked up by passive dns tools then
they can see what we're doing
so let's just do a quick introduction to
passive dns
dns which stands for domain name systems
maps domain names to ip addresses and it
acts like a phone book for the internet
passive dns is a mechanism to store the
dns resolution data and that is
essentially historical data in a
mechanism that can then be queried
so you can go and reference past dns
entries within these databases and
gather information that you need and
there's a variety of services out there
that offer passive dns
services including risk iq and a few
others that we will briefly touch on
later in the talk
passive dns is useful for tracking down
security incidents rogue infrastructure
so for example if individuals in the
team have spun up servers and then
registered subdomains in route 53 and
perhaps appointing them to this
infrastructure and then it sits there
for a few weeks and gets spun down and
you know you can gather information
around oh okay well when did this happen
why was this done and really that going
into a tool like risk iq and having a
look at these sub domains that you will
see shortly can provide information
around that that can be quite useful for
tracking down when rogue infrastructure
was spun up and then asking the
important question of why um it's also
great for tracking down missed
configurations and legacy forgotten
infrastructure i mean this is especially
true when you have mergers and
acquisitions you sometimes end up with
multiple different cloud systems
different services running in it and
being able to go through and have a look
at the sub domains associated with uh
you know
the set of domains that perhaps this
company that you've merged with owned is
a you know a great way of being able to
track down some of this forgotten
infrastructure that perhaps is uh
sitting out there collecting dust and
costing you money
and we can use passive de uh uh dns to
see when domain records have changed for
example when the a record changed to a
different ip address
and it basically works by capturing dns
records and storing them in a database
for reference
so it's
at its heart it's a historical source of
information
it's talking of history
how did passive dns become a thing
so prior to 2005 dns history wasn't
stored and thus was lost whether the
intelligence services had some mechanism
they collected it i i guess is open to
speculation but
that the status quo at the time was that
was essentially lost and this led
florian weimer to invent passive dns i
believe florian wyman is actually an
engineer at red hat now and development
started in 2004 and pdns was first
introduced to the community at the first
conference and you can actually go and
read the original presentation at the
link i've provided here if you're
interested and there's some of the more
technical details in there um so if
that's your thing please
take a look at these slides after the
talk and go ahead and have a poke around
and read because you'll probably find it
interesting
this idea of the dns logger in the paper
we'll just quickly touch on that and the
technical details we don't really have
time to go into a lot of detail but i
think a quick high level overview is
interesting
so passive dns is based on the dns
logger architecture which was proposed
by florian weimer
this architecture consists of sensors
which send data to the dns logger host
and these original sensor
concepts were written in perl and seen
i'm sure the source code is probably
still out there on the internet if you
want to go check it out
um there's an analyzer module which
inspects the data
and then this collector module um which
collects it and then it's stored within
a database and i believe the original
one uh was written in sleepy cat which
is a berkeley db um and i'm sure there's
probably examples of that still out
there if you're interested to
check them out
then there's a who is server connected
to the database and who has clients who
can query the server to gather that
information that they're interested in
by querying the database via the whois
server so it's quite a simple
mechanism on on the surface for doing
this
so
since uh that original concept
there's now many custom many companies
are out there that are offering
passive dns
um that includes cisco umbrella risk iq
which we'll look at shortly security
trails
circle passive dns
i believe that's an api and you have to
get permission to query it but
effectively you can query the circle
passive dns api and gather those
historical data
that you're interested in if you have
access to it spam house
also offers a passive dnf service as
well
so some of these
offer community editions or they also
offer uh free trials so let's take a
look at those should we
free accounts
so there are a variety of free trial
account services out there
this is by no means an exhaustive list
i'm sure there's probably tools that
some of you used out there that you
prefer or other products that i've
missed off here which you wish i had
included in the list
but if you do a google search after this
talk or during it i'm sure you'll find
something uh that that i've missed which
is perhaps even better than what's
listed here
risk iq community edition i haven't
think is great uh it's free
um then we have security trials and spam
house and most you've probably had a
spam house for being the spam photo
company uh they have a passive dns
service um i believe security trails and
spam houses are
trial accounts i've not really explored
them but
um i'm sure if you go sign up and kick
the tires on and you can see what you
think the risk iq community edition is
effectively free it just has restricted
services uh and you have to pay extra to
use the other features that you might be
interested in
you can of course also use archive.org
wayback machine to browse historical
snapshots of websites
so you could go back and look at a
particular website at a particular point
in time if a snapshot was taken to see
what it looked like back then so it's a
nice complement to passive dns
in the upcoming demo we'll use the
community edition of risk iq if you want
to sign up for an account to follow
along with um use the link in the first
bullet point up here it's community.risk
iq.com and then you can
start to kick the tires and have a look
while we we talk about some examples
so gathering information on competitors
let's look at some of the privacy
concerns that you should be worried
about
what does sub domains tell us
subdomains can actually tell us a lot of
information about what a competitor is
up to
and
what we're running as infrastructure or
as products
so
accuracy look at dns records and
something like risk iq
can reveal information such as what
monitoring tools is in place if
somebody's using grafana or or
prometheus
products such as drupal or wordpress
on-prem hosted solutions for example
source control someone might be using
gitlab on-prem and you can gather that
information from looking at
passive dns records
client demo sites hosted on sub domains
so for example they may have built a
demo for a particular customer that
they're trying to sell a new feature to
and that information is effectively
available from the sub domains listed in
in risk iq
new products being tested on sub domains
so this could be
new products that they are selling
themselves or perhaps new products that
they have purchased from someone else
and are running so that that kind of
information can be quite useful from a
business perspective it might be you can
tell if they're planning on launching
something new
if you're a malicious actor seeing that
someone's running a particular service
or product
on a subdomain such as those monitoring
tools i mentioned before cms's can
obviously uh be useful if you're
planning on doing something malicious
and
email services mail servers often these
are listed in there so from a from a
information security perspective these
are types of areas you want to consider
protecting right because if a malicious
act can go in and see these then they
can target them um so apart from all the
usual things of patching and so on and
so forth uh making sure that you've got
suitable security tooling in place you
might want to use some of the techniques
mentioned later in here to just
obfuscate the fact that that you're
running that particular tool
so when it comes to competitors um you
know we're kind of interested in what
customers they have
how can you find out which sale sales
deals for example a competitive one
so we can actually use the information
from the sub domains
to discover what c sales deals
competitor in one
and which companies you know they might
be working with or might be pitching too
so for example we're going to take this
first url here
we can see
customerone.clients.example.com
this actually tells us that this
particular target example.com
likes to list the people that they have
sold products to under this dot
clients.example.com
subdomain
so for example they might have customer
one customer two customer three customer
four customer five and so on so you can
gather fairly quickly their client list
by looking at the sub domains if they
use this format so that's a an immediate
piece of business information that's
being leaked out to the public internet
we can also see perhaps what the sales
team are planning on doing or who
they've been running demos for
so in this example we have
potentialcustomer.sales.example.com
once again if example.com lists all of
their demos under the
dotsales.example.com subdomain then we
can very quickly enumerate a list of the
people that they're targeting
and we can use that information perhaps
uh you know try and get ourselves in the
door there or beat them on our on an rfp
or similar so these are the types of of
just simple business information that
suddenly becomes readily available which
you start to poke around in these
passive dns records on a tool like risk
iq
so these are things to keep in mind and
we're going to probably look at some
examples well not probably we're
definitely going to look at some
examples in a moment of uh where
customer1.client.example.com existed
and then we'll talk about some the
mitigation techniques
compared to secret products so apart
from
who they're working with and who they're
targeting to sell products to
perhaps we can gather information about
what products that they are planning to
release in the near future and perhaps
have leaked that information by accident
onto the internet
so for example let's say we're looking
at risk iq at
and we find that this particular target
has decided that their staging and
development servers are going to be
listed along with in front of it the
particular product that is running
on those that's
server or set of infrastructure so in
this particular example we've got new
iot device.development.example.com
we could imagine you might have new i o
t
device.staging.example.com or new iot
device.uat.example.
but basically what i'm getting at here
is the fact that we can then go in and
query and see that this new iot device
which could be a product name um it
could be you know for example if you're
building a new car or developing a new
fan or developing a new monitor or a
laptop or something like that that
product name dot staging.example.com or
development.example.com is really
advertising that this thing exists now
if you know your competitors see this
and have been gathering other open
source information or perhaps not so
open source information and they can put
together a picture of the upcoming
product that can maybe even get the
brand name uh in advance um so that's
just the kind of thing you might want to
think about
and keep under wraps because you don't
want to accidentally leak that
information before you're ready to
actually pull the plug on going live and
then share the product name and branding
with um with the rest of the world
if you consistently use the practice of
using subdomains for new products
then customers gathering bri can simply
monitor pd and s records and over time
we'll just start to get a picture of
every time you're releasing a new
product you know it appears under this
particular subdomain perhaps my new
product staging
it could use it to their advantage
it's also very easy for those gathering
business intelligence to open a free
account on risk iq or similar and gather
this information without your knowledge
so as some of you may have done well
we've been talking about this you can go
create a gmail account you can then
create a risk iq account or a security
trials account or one of the other
services
and then use that account to go poking
around essentially anonymously a bunch
of domains and seeing what's running
under them uh subdomain list and what
was there historically
so that in mind
you know you should probably think about
some mitigation techniques but before we
get to those
i mentioned we're going to look at a
real world example so we're actually
going to look at a real world example
that was a bit of a scandal because i
think that just makes it a bit more
interesting um
it's a risk iq let's look at this
example which is to
explore the tool and the one we're going
to use is cambridge analytica so some of
you will be familiar from a few years
back the cambridge analytica scandal and
we're going to go poke around a risk iq
and look at cambridge analytica.org and
see what we can find so i'm just going
to switch over to
the web
browser
and here we go so i'm actually logged
into risk iq
and for those of you who are following
along and create an account you can go
in and then you can run a query against
cambridge analytica.org
and once you're logged in you're going
to see a bunch of tabs under here
and you've run run the query um
resolutions who is certificates sub
domains trackers components and so on
and so forth i won't read through them
all but you can have an explore
depending on whether you have a free
account or a paid account then you're
going to get access to different
services i believe sometimes some of the
services they have available for a
certain amount of time for free just to
give you a chance to take a look at them
and see if you're interested before they
uh encourage you to pay for the uh
the full version
what we're interested in is the sub
domains
so if we take a look through these we
can see some pretty interesting
information right off the bat
we can see that as i mentioned some
companies like to use this client
subdomain so we see
clients.cambridgeanalytica.org
immediately that says to me perhaps
we're going to find something underneath
here such as
client1.clients.comgeneralistic.org we
could start to enumerate who they were
working with or perhaps who they were
attempting to sell services to
other technical information about their
infrastructure is also available
it appears that they had cpanel
i think when we look down here we can
see uh perhaps kubernetes was running um
i mentioned before about mail servers we
can see an nginx ingress point uh to uh
kubernetes
and here's a really interesting one here
we've got clients.cambridge
analytica.org and we've got nyu langon
so this looks like it was potentially a
client or somebody that cambridge
analytica was attempting to sell
services to
and at the moment we're going to go back
and look at some
research
just to you know tie these two things
together and see if hey was this right
is this actually a client of cambridge
analytics or someone they attempted to
sell something to or was this just a
coincidence
so let's just take a look and see
if perhaps anything's still running at
nyu langone clients.cambridge analog to
get up
oh okay
and no there's nothing there but
cambridge analytica.org is still up
which i was very surprised about
um
looking at this though i'm not entirely
sure who's managing the site it does
look a little suspect um
i don't think this graphic here was
necessarily something that the cambridge
analytica team would have used as an
advert
uh and then the more we look down you
can see that those broken image links um
i don't think some of these hyperlinks
work and i'd be cautious before clicking
on too much on here so i'm not sure if
someone's come along and purchased the
domain name after they abandoned it and
is now
running something else behind it but
it's quite interesting to see that this
is still up
so let's take a look
at the slides again
so i mentioned cambridge analytica and
we saw that sub-domain now
security researchers were able to probe
active and passive dns records back when
cambridge analytica was in business and
i don't think they are anymore and
gather quite a bit of information around
them um including these customers and
sales targets
so we thought nyu langone.clients.com
analytica.org
so in fact we do a
a little bit of research we can see an
nbc news article from back in 2018 and
within this article they actually
discuss
how there was some consternation between
nyu langone and
cambridge analytica
whether they were clients technically or
whether they were being pitched services
to there was certainly some um
i guess
problems there and uh it blew up in the
press um so
i think
evidently this is an example where your
your product name or your company name
being associated with another company's
uh perhaps something you wanted to keep
under wraps is it has ended up actually
having some negative press
so that's a a separate point which i'm
going to touch on is kind of closely
related to this whole um don't use your
client name as a subdomain but there's
also that you know the other problem is
do you have control over what other
companies are doing when you're
purchasing services off them and if
there's a scandal associated with them
how do you avoid uh not getting caught
up in it and ending up with nbc
reporters knocking on your door
so that was just a quick real world
example uh and you can do some more
digging um unfortunately we don't have
enough time to do that but i think it's
it's quite a fun exercise
submit irrigation techniques you know
what can we do to plug the leak how can
we avoid being a victim
of someone stealing information about
our upcoming products how can we protect
our customers by not accidentally
leaking information about their upcoming
products when we're working with them
how can we
not leak our sales list and so on and so
forth
so of the skating sub domains so i like
this little quote here at top my
co-worker richard had had used this um
somewhere and i borrowed it and had
rewritten it slightly but he said
remember resolutions from requests end
up in public dns even if the hosts are
referred to a private or firewall
so a number of techniques we can use to
ensure we don't leak information
first
so obvious one is don't use
client.salesexample.com
or clients uh
customer.clientexample.com naming format
so if you saw with the cambridge
analytical website you know
if you put your customers out there uh
as a
you know
nyu langone.clients.com
you're kind of exposing them to the
press if something goes wrong i i'm not
sure cambridge analytica realized that
at the time um
we'll let the
journalists do their jobs there but i
think you know obviously there was quite
a bit of embarrassment there
um don't use
clients.sales.example.com because really
you're giving away who you're planning
on trying to sell services to so these
these two methods should be avoided they
seem so obvious when you're setting
things up oh we'll just group all of our
customers under client.example.com but
of course you've accidentally then just
given away a load of really useful
information to competitors
and obviously you can sit sensitive
information such as your proof of
concept websites and products and
services behind a vpn
and use internal meaningless domain
names for example
cabbage.example.internal
you know use private zones and avoid
public dns if you keep everything locked
down internally
and give it obscure names that only
people within the company know about
then that can help immensely
if you do need it to be publicly
accessible you can obfuscate subdomains
so in this particular example we've got
chicken chicken.example.com
so we could use a a mapping scheme on
the back end that only people in the
company know and of course the the
customer itself if they have to go visit
say a uat site
where egg is the equivalent of client
and checking is the equivalent of the
client's name and you know you could use
any number of file to represent
whichever customer uh you need to but in
this case
client.example.com could be you know
customer customer.clients.mywebsite.com
um and really only your team knows about
that
another thing you can use is an anagram
generator to create subdomains so thanks
to my colleague richard bullington
maguire for this one uh who's quote i
stole at the top there
um
wordsmith.org anagram it's a great tool
you can basically put in a word and use
that as a seed so come up with a initial
phrase
use that as the seed and then from there
create a set of just random dem uh
subdomains and then sub sub domains
which are then used to host
infrastructure on that is not production
infrastructure uh but perhaps has to be
available um you know publicly for some
reason or other even if it's locked down
behind passwords and
and login mechanisms
so we've done this in the past ourselves
uh with working with customers when
they've had a product and there has to
be a staging or a ut uat website we'll
use a tool like this to come up with a
set of
random names that are really meaningless
to anyone that's poking around
but on the back end there's a mapping
like i mentioned before the
chicken.example.com
and that mapping uh is very obviously
very useful to the engineering team and
to the
you know the business unit that's uh
working with this particular product
before it becomes live but to anyone
else poking around as uh yeah you're
just not gonna know what it is and you
can also use that for infrastructure as
well so we saw cpanel and we saw
kubernetes and we saw mail and some
other tools it may make sense in order
just to obfuscate and just make general
like reconnaissance of your site more
difficult
using some random names and subdomains
and and then really what that means is
anyone poking around in it they may take
that link and go explore it but if then
they're
blocked by whatever mechanism you're
using for authentication and
authorization it's going to reduce the
amount of information they're getting
especially the login screens
particularly non-descript and doesn't
give away what the actual uh services
that's running
might be overkill for some of you but
it's something worth it considering
anyway
then finally you need to train your team
to think about security first when it
comes to business intelligence this
isn't just a job for
the engineers it's for the whole of your
development team the architects the
engineers and the
product owners so really getting
together coming up with a mapping scheme
thinking about the security first
approach of baking in the obfuscation
of you know the product name or the
client list so that when you're
basically building sites out you're not
leaking that information
so whilst
security by obscurity is not the best
approach to the vast majority of things
as a link in the chain it can be quite
useful when it's used in the right
fashion
so in conclusion
what do we learn passive dns is useful
for security researchers it can be great
tool
um you know if you're using it in the
right fashion for understanding what's
going on with your own infrastructure
and i'd recommend if you're interested
in reading a bit more about you know the
use of passive dns for security um
it's also unfortunately useful for
competitors gathering business
intelligence on you so keep that in mind
as we've talked about you know
throughout the last few slides and we
saw with the example from risk iq um
that information is all available to
anyone who set up an account so
best to avoid putting it in there in the
first place
think about what you're leaking through
your dns entries so as i mentioned the
customer list sales demos and of course
also think about if you're purchasing
services offer a third party and they're
including your name within this
structure of
yourname.client.example.com that could
be a problem also consider the fact that
some of your customers you may have you
know legal reasons why you're not
allowed to list them um so avoid doing
that and you know avoid getting into hot
water on your part and if
third parties are putting your name in
and there is some sort of scandal um you
know you want to try and avoid that if
you can
come up with the policy and process to
avoid what we've mentioned before so
that was really the last point i
mentioned on the previous slide it takes
a whole team to build a product and take
the whole team to defend it really
security shouldn't just be
you know one team's responsibility it's
the whole of the
whole of the group that's responsible
for a particular product or service
you know an educated team so watching
this talk would obviously be helpful
there's lots of great resources out
there probably a lot of them much better
than my talk uh on why you know you
should use obfuscated subdomain names
for products and other techniques you
can use to help protect you from
from
leaking information but educating your
team is a great first step
so with that said
the q a session will be starting
momentarily over on discord
i'd like to thank all of the folks from
the recon team uh once again for
putting this on this year because i
understand it's been a great challenge
with covid and uh you know having to run
things online and not being able to do
things in person so so thanks you're
doing a stellar job and for anyone
looking for a job please check out
moduscreate.com that's my final plug
today we've got loads of open positions
and we're always recruiting engineers
and security folks and devops so please
go check it out
and with that said thanks once again and
i look forward to speaking to you on
discord bye