yeah so uh as mike was talking it's kind
of hard to
manage access to cloud and see who has
access and everything so in this talk
we're gonna talk about kind of like some
of the security best practices that i've
learned working at microsoft and then
like the tool that my company created
so we're going to start with a little
bit about myself
so i'm an identity security engineer
with years of experience creating
identity and pki services for microsoft
and then after a few years decided to
leave and co-found ketos where we create
pki services easy to use pki services to
kind of make the world a password-less
place
uh and then there's my twitter handle if
you have any questions feel free to send
them there and also send them
in to the cloud village and we'll answer
them at the end and here's a picture of
me outside uh since i didn't know we
were gonna talk about rowing i wanted to
show that sometimes i go outside too
so uh let's start with something that we
can all agree on humans suck like
they're slow to respond when there's an
outage they get hacked because they
don't follow your recommendations and
they can't scale but they
they don't follow your recommendation so
they will still go on facebook they'll
still click that phishing email and why
because we're humans we make mistakes
they can't scale we we can't go all the
way
to cloud scale and having hundreds and
thousands of machines as mike was saying
like automation is taking over and we
should take over uh it doesn't make
sense to people still going in and
clicking through and creating vms so we
have to
try to get them out of the equation as
much as possible but also understand
when we're designing our security policy
they're not going to be perfect humans
having
doing everything perfectly like
count that they might choose a bad
password and help them do the right
thing the easiest way
so in this talk i'm gonna start by first
talking about like kind of like the
cloud best uh security best practices
i'm gonna go through each one of them so
i'm not gonna read them out right now
so first one is keep management identity
ideally to zero so uh mike in the
previous talk was also talking about
pim and adjusting time access
so um
it's something we're going to talk about
but the first thing we have to do is
infrastructure as code
if you don't have the humans go in and
click through and everything and they
don't really have to go to the portal as
much it will make it easier for them to
not have access to
to to the infrastructure and
harder to hackers to get in when humans
don't have access and the the way to
make this is making it easier for them
so one of the things we have learned at
ketos is that one of our most visited
sites is our docs page where we talk
about how to deploy
cloud vms with already ssh certificates
in them the reason for that it's easier
to just go to one site that has all the
information than to have to go and
figure it out and if that information
has the best practices then your users
are just going to follow the best
practices so encourage
arm templates server form templates
blooming templates whatever your
organization uses to make it easy for
the user to do the right thing maybe
create some with the ports closed and
all that
stuff and then as i was saying about
just in time access so one of the things
that justin time access gives you is
first of all you don't have access all
the time so it makes it a little bit
harder to scan and it also gives you the
ability to have dual key approval dual
key approval helps because then you need
another account compromise and you need
to convince someone at microsoft we had
to do it that like i would go in person
or like call my manager and be like hey
i need access to this please approve
this which is very hard for a hacker to
be able to uh
imitate that so
uh
having dual key approval having some
policy that the user has to actually
talk to the other person will really
help mitigate some of the attacks
and the other one it also makes it hard
it makes it hard to get into
the infrastructure forcing people to to
do the first thing that we were talking
about infrastructure as code and
automate some of the stuff if it
is if it's going to take you a while to
get in every time you have to get in
then you're going to find ways to
automate yourself out of it
and then
we we have talked about uh how we're
trying to get out of it but you also
have to have a strong identity so one of
the things
that we do at ketos and many companies
do as well is having a completely
isolated identity
that your email identity is going to be
compromised because you're checking
email you
you'll get phished and
bad things are gonna happen to that
account if you use it a lot you enter it
for multiple sites so the user is kind
of like train
a password and and move on with their
life
so if you create an isolated identity
that you only use for production so
you're only using it in aws or you're
using only using it in azure that kind
of helps your users
only think that that identity is for
that and it's harder to get fished
so something to keep in mind that you
can have two different identities
and those identities should have mfa
nowadays with cloud there's no reason
you shouldn't have mfa i'm a big fan of
fido and piv and hello for business
because i worked on some of those
technologies or because it's the best
way
to work with
for humans like it's way harder for a
human to remember a 20 character
password and have
400 different passwords for all their
accounts uh but it's very easy for a
human to
grab their ubi key and plug it in
and get access to
enter a pin and get access so it's way
easier to remember
and it's a better user experience i
actually prefer having this and typing
six things instead of having to have a
big password and cell phone
and the other one is you can do
conditional access that i'll cover a
little bit more in another slide but
basically
it allows you to have
if the user hasn't logged in in a while
like ask for mfa although i recommend
asking mfa for each time and if the
computer is not compliant you can
block the
the access
and then as i said humans are always
going to try to find a way to get in
late the lazy way and one of the ways
that they do that is creating a service
principle a service account that with a
password and then they can set the
password whatever they want so if you
set up all the mfa and everything they
can still set up a password and bypass
that so have some policy that you cannot
create
pulls with passwords and also you
shouldn't use passwords for service
service
certificates are out there use pki
and the last one i want to talk to
is remove vm password so
most vms
in cloud like aws and azure and jcp will
come with a default password that you
set and it's like a local account
make that automate that in your uh
in your scripts when you're creating
that it's just a random password that no
one has access to
and don't use that password the reason
for that is you don't want to have a
shared account and also the engineer
that is creating the vm will have that
password and if they leave or in the
insider thread like they can still
recommend using
your azure active directory passwords or
basically your they need to provide your
passwords to access your vms and there's
a lot of tools that do it like
uh you can domain join your machines and
for linux there's many tools we actually
create in keto is one of those tools so
there is a lot of tools out there that
you can use
and then uh the other one is keep your
network closed i don't know why
everybody decided it was okay to open
all the ports uh maybe it's
about earlier that
it's just people don't
like before it was like the network
engineer making sure that the firework
firewall and everything was close and
now everybody has access
but
from going from like completely a
firewall to being completely open we
should still keep just necessary ports
open be very strict about your ip ranges
so no your whole corporate doesn't need
access to manage your vms like your
management board should probably be
closed all the time
and maybe have some ip addresses and
same with service to service
authentication not all services talk to
each other just have the specific ip
addresses of the services that talk to
this specific service have access to the
service there is no reason to have a
completely open network
and then networking jits so that goes
back to the
management ports
so azure offers is networking just in
time access which means that it when you
need access to the port is always closed
no
no port no no ip has access
and
okay sorry
no no no port has access to
[Music]
to the machine and then uh
when you need access it will just
request access for you
i got a hurry up here so i'm gonna keep
keep going
uh the strong device so a strong device
so you manage your identity you secure
your identity you do all this great
things for
keeping your identity secure
but then if you have a device with
malware they can still steal your token
and do stuff like that so you have to
have your device managed make sure that
they get scanned and they have proper
policies as you can see for the medium i
prefer intune is way easier than ad gpos
and it's cloud-based and easier to
manage and you can use conditional
access based on this device health so if
the device has been scanning everything
they give you access if that doesn't
it'll block access even if you have the
correct credentials and then
generally we use isolated device for
production so this is something that
some companies do it's kind of going the
extra mile
um but we're an identity security
company so we go the extra mile that you
have a completely different device that
you only access production with so you
have your dev machine that is where i'm
presenting this
you have your email and everything
actually touch the production you have
to open another device and log in with
production identity so your production
identity is always in that device so a
compromise to this device wouldn't give
me access to the device identity
and the last one is use
cloud tools so you have the azure
security center that will help you with
compliance and help you with some stuff
when you're moving to the cloud that you
might oversee like hey you left all the
ports open you should probably close
some of the ports the other one is
managed service identities it allows you
to
manage your identities for you and this
is all azure
i i mostly use azure so all my
terminologies in azure but most of these
cloud providers have the same like so
you just have to translate it to your
cloud provider so manage service
identities will the cloud provider will
manage the password for you this makes
it hard for a user to get at the
password and use it and it also avoids
outages such as
bad password rotations or anything so
using manage identities is great for
availability and it's great for security
and for the passwords that you can use
that you should use azure key vault kms
any of the password vaults that will
allow the services to access the volts
and you can rotate them and have all
your policies to to do all that stuff
and and once again if you make it easy
put it put some code samples out there
and be like hey this is how you get your
passwords from a key vault or this is
how you get your passwords for of this
how you authenticate with the managed
service uh service identities it makes
it easier for your developers to use the
right tools and instead of them having
to figure it out on their own and also
makes a better uh experience for the
security team
i always thought that security teams
should
people should come out with security
meetings instead of with more tasks to
do with ways to improve so they look
forward to those security meetings
instead of dreading like oh i have to go
to another security meeting so let's see
how we get delayed this time
and then i already talked about pim
networking jit and intune so those are
some that i already covered in the talk
and then monitor everything
so you should have your sim up and
running with alerts uh azure security
center it has some alerts but
and then turn logs on every service even
if you're not reading all the services
you have to turn on the logs just to
have forensics if something bad happens
you want to be able to go back and look
so
don't save those extra few dollars turn
on logs on every single
you run
and also run your own custom alerts and
code because some of the things they get
through infrastructure but one of the
things that we did great at microsoft
was we had some trainings that said
everybody had to do them and we would
talk about security the red team would
go up there and say like hey this is how
we get in and this is how you guys can
stop us so then like the engineers are
aware of it and you write um
you write different
alerts and here i have an example of one
that we actually write uh we have i'm
not going to show exactly what we log in
where this is but somewhere in our code
we log this and the engineer on call
will get a call saying like hey someone
is trying to mess with our apis let's
track that
so
basically if you follow this we solve
security uh you will never get hacked no
i'm kidding
this just is not a silver bullet these
are just best practices and that's the
thing even if you follow all this you
can't
make a mistake something's gonna happen
and a hacker can still get in
so
i'm going to talk about some of the back
doors that i've seen people do and then
i'm going to talk about cloud watcher
our tool
that uh
that we have created
so the first one is adding
so if you have your groups that are
echoed like you might have like your
engineering group and that group gets
hackled into your subscriptions if you
add an extra member in there it's a
group that you don't check the
membership that often so having that
extra member will give them access to
all the stuff your engineers have access
but you won't notice
having an administrator so this is once
again azure specific classic
administrators was a way of doing things
like four years ago we have been trying
to facing them out since then
and
now they they have moved them to another
uh tab which is great less chances of
people creating them but it's also
harder to detect when someone adds one
so if like a hacker adds one it's harder
to detect that there's one than that
they have uh
another one that i used to love when i
was doing pen testing is adding a
service principle as a contributor to
the subscription and if you put like the
player somewhere in the name no one
wants to break deployments so no one is
going to touch it so something to make
sure that you're keeping track of and
usually deployment has contributor
access because you have to create and
delete resources this one is more
developers being lazy that it's um
removing the azure active directory
authentication only so
they might want to create a password
for sql and just use that instead of
their active directory account so having
that checked uh forces them to use azure
active directory
uh add more
permissions to ikv
so um
this one is if you have all these
different permissions like let's say a
developer only has list permissions but
it doesn't have get if they get the get
permission now they can steal the the
code uh the
the secret and use it so keeping track
of that it's kind of hard because when
you're looking at the keyboard you're
like yeah they should have some
permission so that's about right then
adding ip addresses to a firewall this
one is one
match when you have like a long list of
ip addresses and you just add one there
might be a developer being lazy maybe a
hacker from
else trying to get in so
catching one is kind of hard if you have
uh
if they try to remove the rule or
something it's easy but catching one is
kind of hard for a human so we created a
solution that kind of
monitors all those back doors and
it's basically a powershell model module
that runs in azure automation you can
get it from the azure powershell gallery
it's all free we basically used it
internally and decided to open source it
like we this is not one of our products
or anything we don't make any money out
of it we just wanted to share with the
security community
uh scans your subscription and detect
any of the configurations changes that i
mentioned
and alerts you if the change has been
detected so here you can see no changes
have been detected this is kind of like
a run that um
would be fine
and then here we show one that it hasn't
been detected and we have it that it
fails the runbook whenever something is
detected and then we set up an alert
that when a round book fails it calls
our on-call engineer so here you can see
we added some stuff and it failed
so what it covers our back changes so if
something gets added to the r back or to
a uh
it will detect it and let you know
because production access should be very
monitored classic administrator changes
so we'll alert if that classic
administrator is changed azure resource
provider uh changes so this one is uh
azure
resources get access through azure
research providers and you can and they
get kind of like automatically added to
your subscription but you can create
custom ones so for example like one of
the ways that i like explaining it is
like the azure resource provider for key
vault has access to the vms so they can
encrypt the disks so it can give you
some access to the
resources so we just like to keep it
clean so we added it for monitoring
research creation and deletion
this might seem a little bit harmless
but if someone adds a vm that then
they're using for tests but it's not
production network and then that doesn't
get patched it could
it can be a way in later so it's
important to keep everything clean and
also if someone deletes something
in a production subscription you should
be alerted because you don't know if
it's something that happened by mistake
and all of a sudden you one of your
availability zones is down and you'll
eventually have a disaster or something
so it's important to keep track of that
change of group members risk there we
only do nested groups uh sorry we don't
do nested groups so we only do first
degree so any group that is anywhere in
your subscription will monitor the
membership to make sure that nothing has
changed
sql firewall changes so if a firewall
rule gets added a single ip or something
will
do
sql server aad admin changes
so as i mentioned if they like uncheck
the aed only or they change the admin
group or anything
we'll alert azure keyboard uh access
policy
so once again
uh those are the ones that i talked
about that like changing the how many
permissions you have into the
azure keyboard firewall changes
so
i'm going to address the elephant in the
room maybe of you might be thinking like
he already talked about cloud tools why
isn't he just using azure security
center and like sentinel and the reason
is like those
don't cover all this stuff so we use the
all of those
plus this
so we have all our detections and
everything but this is more kind of like
making sure that nothing has changed
because azure security center it's not
going to alert you if someone adds an
extra
person to a group that seems pretty
normal or if someone has an extra ip
address that's something that my
service got a new ip address or
something so
this one we keep it just because we want
to make sure that everything
we get alerted and we know if something
changes
so now i'm going to talk about how we
set it up uh this is a great picture of
how we actually set it up
and then here i kind of
make it easier uh we just have one cloud
watcher in this one but every
subscription would have a cloud watcher
watching in the other subscriptions
and you have your production
subscription in this case that is the
one we're watching you have a cloud
watcher in this subscription that
ideally has different identity
management
than the original so like you don't have
the same administrator so a compromise
and one identity doesn't compromise the
other one
and then you have the baseline in a
different subscription so as a
compromise in this one doesn't
compromise the baseline which this one
only has read
we actually keep this one in a different
tenant a whole different setup um
but so
yeah so this one will read the baseline
which is basically a json file and then
it'll compare it to your resources
so what's next the next thing we want to
attack is uh create a ui to visualize
our resources and kind of like alert on
certain things
that might be
a vulnerability
so
um
like
you you we we would show kind of like
all the
resources and say like hey like it seems
like this is very open you should check
that and that that will help us because
we think
security um
security reviews
don't
like all diagrams so like having
something up to date that you can
actually use for a security review and
actually
being based on the scan of your
subscription instead of what you think
the service looks like
will help a lot with the security then
add more resources as you saw right now
is kind of limited but we're a small
company that doesn't use many types of
resources so that's one of the reasons
we're open sourcing it if you want to
use it and you want to add your own
resources please feel free to make a
pull request and not them and then the
last one is we want to integrate with
pollubi terraform and
all the other providers to be able to
match the baseline also to your
infrastructure as code to make sure that
nothing is changing from your
infrastructure infrastructures code to
the actual
console
and once again thank you so much
here's the link to the
uh
to the github project
here are the slides and i actually also
created a set of videos so make sure to
check that out i'll just go through the
whole things of how to set it up
and that's it i don't know if there's
any questions jeff