hello everyone this is scaling apsec
through education
just briefly touch on
who we are
secure delivery
um some takeaways from the slide here is
that we do this sort of thing
commercially at scale with large
companies and and small
but it's all built on os projects and
especially built on the
open application security curriculum
of which i am one of the project leads
together with toby irvine
i also lead the os cornucopia project
together with colin watson and i'm
currently on the os global board
lots of o-wasp there
the real takeaway from this slide though
is my handle rewtd root demon under
which you'll find me gooning here on the
defcon discord
you'll also find me under that handle on
twitter and on linkedin
my dms are open so please give me
feedback on the things that you hear
today and the things that you liked and
didn't like
scaling uptick in large organizations in
very little time
here we go
to start thinking about application
security we have to define it let's draw
box remotes that were clear
cyber security is incredibly broad i.t
security covers corporate identity email
security desktop os's
server patching file shares printers
especially recently
and there's a lot to do there
we're concerned today with the part of
cyber security you have to deal with
when you're building your own software
systems
the part of cyber security that this
village is all about
your systems can be secure as designed
and built but there are also many ways
in
how you are building them that can screw
you over from a security point of view
now someone could push production
passwords or cloud api keys to a public
repository
or a sketchy person can approach one of
the developers on your team outside of
work and offer them an eye-watering
amount of money to a little backdoor
a supply chain attack an open source
library that that you happen to use get
some nasty code introduced to it
you put it in on the next build of your
system
and there are many many more
we're talking about
quality supply chains materials value
and activities and design and
manufacturing processes
then what we really mean by application
security is
product security
now that's all pretty clear
but security is an intrinsic quality of
your system sounds a bit philosophical
so let's define that too
this is the iso iec 25010 system and
software quality model
it's
in left to right importance the quality
model is concerned and connected to how
a system provides value
so you might think that reliability is
more important than performance
but a system with five nines
availability that's really slow and
practically unusable by our customers
completely useless
now security sits somewhere on the
border between the non-functionals that
people care about and they can see
and the non-functionals that people
don't much care about and can't really
see
compare for example
the business impact
of a change that was pushed to
production that adversely
affects performance
with a similar change that's pushed to
production that adversely affects
security
the former
your customers will notice it
immediately
the latter
well something bad might happen
it might never happen
but the famous saying hope is not a
strategy
when deadlines pinch or resources get
limited
work on these qualities will drop away
pretty predictably from right left
ultimately just leaving
functional work and testing
this in addition to the team simply
being unaware of the requirements for a
non-functional
is when technical debt starts to pile up
when you hear technical debt think
lower quality
so product security is one aspect of its
overall quality how do we ensure that an
appropriately high enough level of
security quality
while still delivering at the pace
required
now this is a common way to think about
maintaining security where rapid
delivery of changes to production is at
one end of a scale and being secure is
at the other end
and as an organization you
kind of pick
where along that scale you're
comfortable with being according to your
risk appetites this makes intuitive
sense to us because as humans we
perceive moving quickly as carrying more
risk like driving a car with your foot
down
the problem with this intuition when
applied to application delivery is that
it's completely wrong
i'm not saying rush everything it'll be
fine
but rapid delivery and high quality are
not opposed to each other in fact they
complement each other
having the ability to develop and
release change to production quickly
means
that you can improve things quicker you
can find and fix issues quicker you'll
find and fix those security issues
quicker
it means you have automated deployment
processes
which remove human error removes
opportunities from licious intent and
increases the reliability of your
ability to actually deploy
if you have automated testing processes
then you're increasing your functional
correctness and you'll also be
increasing any un non-functional
qualities that you're directly testing
for think of performance again
hopefully security
you've probably got code that's clear
and straightforward to extend
and you're not handing off changes to
other teams for assurance or for
deployment
you simply can't deliver pace unless
you're paying attention to the
non-functionals performance
interruptable reliable usable
maintainable and secure systems can be
changed with confidence
low quality systems are harder to change
they're brittle unmaintainable
unreliable
insecure slow
these low quality systems can't be
changed quickly they'll functionally
fall apart if you try and everyone
is quite correctly far too scared to try
every change leads to unpredictable
failures and overly complex systems
the slowest release cadence projects
that i've worked with have all been the
lowest quality the worst designs and
terrible code bases
delivering
slowly isn't more secure it's fearful
and if you're afraid
of changing a system then that system is
not secure
taking all of this into account a
security incident is a trailing
indicator of low software quality
a problem set in long before that
incident happened right training
indicators confirm long-term trends
but they don't help us to predict them
they're a terrible measure to base your
organizational decision making and
budget allocations on which is why a
massive internal re-budgeting and
shake-up in response to security
incidents
usually a bad idea
so how do we manage security quality
when the only thing we have is a
training indicator well we need to get
some leading indicators of software
quality to get ahead of any potential
instance if we had leading indicators
that we have good measures to base our
decision making on
the leading indicators that are used
to define
high performing teams are these the four
metrics that matter
so high performing organizations produce
derive more value from high quality
software delivered by their high
performing technology functions
analysis by nicole forsgen
gene kim and jess humble in 2018 in
accelerate
shows that there are four metrics that
are strong leading indicators of high
performing organizations they are lead
time deployment frequency
mean time to restore and change fill
percentage
notice that functional correctness
doesn't impact any of these right you
could build a completely
wrong system
make no money
mess up your customers
with extremely high quality if you like
so
getting the functionals right is still
important
but non-functionals do impact all of
these
security is interesting though because
improving the security systems doesn't
improve any of these metrics
except insofar as we also improve the
other non-functionals
how we improve product security can make
these metrics worse though
we need to aim for as minimal an impact
on those metrics
while improving security and that's why
tracking these four metrics is important
for security
so if we're measuring these four things
in delivery
then we have direct indicators of the
underlying non-functional qualities of
our system if these are trending badly
as a team you're not paying enough
attention to the non-functionals
these are trending well then your
current balance between getting features
out the door while maintaining the
quality of your work is sustainable
to hone in on specific measures for
security
we're going to need something more
though
so in this study of
the root causes of all cves reported in
the eight years between 2008 and 2016
says that almost
all are in code right in the
implementation section
this makes sense because there are only
so many ways you can scrub a design or
configuration
but there are countless ways to scrub
while coding it's extremely complex
traditional focus cyber security design
reviews configuration hardening sys that
sort of thing
sits in this area it's useful but it's
far from the whole picture
so if most of product security lies in
preventing security defects and code
from reaching production and in fixing
the ones that are already there
and having a measure of direct security
defect density and code from a security
testing tool is a good direct metric
for security quality
lastly
we've seen that we're building how we're
building a system also affects the
product security quality
so we need some measures of that so what
are the activities that have positive
effects on quality generally
well let's go through the ways that you
would ensure performance in your systems
first
as it's more familiar and then we'll
take it from there to security
firstly we'll think about the
performance impact any change
might have while we're planning for it
right
during sprint planning
we'll plan our overall system
architecture around it being performant
we'll make sure that it is performed
we'll ensure that everyone on the team
knows how to write performant code
using the languages and the frameworks
that
need to happen
we'll hire people with domain expertise
and performance to come in and coach and
guide our teams
and the wider business to understand how
important performance is
will peter review code code changes for
those inevitable human errors that slip
in that might impact how performant our
code is
we'll share known performed code
and internal libraries across the teams
we'll make performance a key
consideration in the selection of any
third-party libraries or third-party
services that we depend on
we'll automatically test all changes for
performance regression on every commit
to develop so that developers can catch
things quickly and catch them early
we'll prioritize performance fixes
that are on our backlog or in the
development cycle
and we'll inject comprehensive telemetry
to measure and monitor performance in
production
we'll investigate any performance
incidents we have in production and we
will widely publish the findings
and we'll have responsibility and
accountability for performance will sit
with the team the people building the
systems
now
you've probably already guessed the big
reveal is going to be here
that's right
it's similar across all of your
non-functional qualities right so
security is going to fit in exactly the
same place
now some of those we can measure and
make visible like we can have a record
of developers who've had secure training
a team with only developers who haven't
had it is much more likely to create a
system with low security quality to
given
we can check github and gitlab for
evidence of pull requests and reviews to
show that peer review is actually
happening
now we can't see that it's for security
but we can still see it is happening
we can check
jira or whatever work tracking system
you have for the age of items that have
been marked as security work to show
that known issues aren't being
de-prioritized they're not being ignored
and we can check the build and test logs
on our orchestration platforms for
evidence the security tool test runs
and we can pull all these
delivery metrics direct code level
security testing output and evidence of
good ways of working together create
strong
indicators that we have a product
security under control
but how do we wrap this up in a scalable
process that ensures security is at the
level that it needs to be at while still
delivering at pace
we take a page
out of site reliability engineering
right we have some experts in the field
that build
self-service platforms it's important so
they get out of the way
and then
provide us with launch engineers that
work closely with the product teams to
make sure that it hits the ground
running it has all the things necessary
to be reliable
service level objectives well it costs a
lot more to run a five nines reliable
system than a two nines reliable system
right
so the difference between seven hours of
personal downtime to just 26 seconds
so agreeing what the product requires is
fairly essential
service level indicators relatively
straightforward reliability right is the
system up or down
how long has the system been down for so
far this month
the operating model is simple it's data
driven so there's no emotion provides a
self-regulating back pressure on the
product team if they let reliability go
down with whatever changes they've made
to the system over time
a product team that has production
support responsibility handed back to
them
when then has a strong incentive to now
focus on getting reliability back to
where it needs to be
sre won't take on production support
again until reliability is demonstrably
back to the agreed level
means the developers are not building
new features because they're spending
time doing support
and they don't particularly like being
woken up at 3 30 in the morning either
can we apply this method to security
product security engineering
so you can map sre to security it's the
same just making security part of the
engineering function now it's an
adjustment
for some security teams to become that
internal consultancy with an education
focus
who are building pipelines and making
sure there's automated tools in place
but others are already doing this right
a lot of organizations are already doing
it and a lot of those organizations are
also already grouping systems based on
criticality with greater security
requirements on those critical systems
so they need to specify where the
security level indicators need to be for
each group but they're almost there
already
now we don't have one
clear service level indicator equivalent
so we're going to have to combine
multiple indicators to create a
composite sli as you've already seen
now back best to provide that back
pressure is going to vary from
organization organization
we've made security visible through a
slice but without meaningful back
pressure the product teams can't
self-regulate for security
unlike reliability
you probably aren't going to get a
series of smaller security instance
leading up to a big one right failure
mode security is immediate ish not
gradual
this month we've been hacked twice
briefly already we need to ease back on
feature work and work on security so we
don't get hacked anymore
it's unfortunately not a realistic
situation
so strong incentives for product teams
uh need to be in place to keep leading
indicators where they need to be they're
essential to avoid security quality
deteriorating
which will lead to sudden critical
failures
you need to be able to answer the
question who's contributing to the
system code design product decisions
what subsystems make up the system what
components make up those subsystems
which code repositories map to those
components from the bottom up you need
to be able to go from source code
repository it might be training a
problem to the components subsystems and
systems that are built from that
repository
without asking people
people come and go
software tends to stick around probably
longer than it should if you can't do
this from an engineering point of view
the organization has zero chance of
getting its
[Music]
act together
or being able to build high quality
secure systems or respond to identify
problems adequately and evidencing to
regulators that things are under control
will be very hard proving there's been
no negligence the event of a security
breach will be next to impossible
security level objectives can be as
simple as comprehensive as your
organization needs and automate it to
the degree necessary to scale to the
size of your organization
this is a representation of the system
that's currently previously built
for a large organization it's heavily
automated integrated with multiple other
systems such as active directory hr for
training data
their eim system github enterprise
with a composite security capability
score calculated by an algorithm which
factored in all the security level
indicators identified as important to
the bank
the slo system itself exposed an api for
automated deployment or change control
systems
to check the product and its team were
currently in good standing to allow for
rapid deployment to production
now
you may not need all this but the act of
gathering requirements and establishing
the baseline is very valuable and brings
all stakeholders in your organization
together to agree on the right balance
and the approach to security quality for
you
we're all competitive
no team wants to be the worst performing
quality assurance activities have to
happen
so if a delivery team does them and
evidences them they can go straight to
prod
the carrot right
fast track to production
if they don't then someone else will
have to do it and that takes up a lot of
time
as we saw earlier
you know you have to get the right
operating model to acknowledge the
aspects of product security explicitly
to measure them to make the right people
accountable
so
there will be many talks there have
already been many and there are more to
come
on most of the rest of the slide
today i'm going to focus on the part in
bold mostly because i know that without
developers learning to write and then
continuing to write secure code
we cannot hope to succeed
my talk for oas but black at eu last
year was on exactly that topic
r u b f g red unless blue finds green
so i'll leave the y
as assumed for this talk and i'll focus
on the how
but feel free to check out that talk
afterwards if you disagree with that
premise
and feel free to comment and give me
some feedback on it as well
so this brings us to secure software
development training the how of it all
today i want to talk about our latest
our being oas latest attempt because
this isn't the first time we've tried it
to standardize and codify a curriculum
for secure software development
this is an awesome initiative i'm a huge
fan of os
especially of the open source approach
to building stuff
because then things happen in the open
and everyone
anyone can contribute
it also makes sense to use the os
projects that set the standard for
measuring software quality just the svs
and
the deployment maturity which is the sam
and to use the great work that's been
done on these we also leverage
cornucopia because that leverages those
same projects as well and covers design
principles
and the top 10 because that's where
everything starts
speaking of starting let's kick this off
with the fundamentals the top 10.
the top 10 although there are now
several
is where most teams start to look at
product security that's from the common
defect side rather than how to build
things right
but the original top 10 the one that's
being updated this year and is
referenced indirectly now by the pci dss
despite
not being a standard
is still a good place to start
henry van stock
who's the current top 10 project lead
and executive executive director for
royal wasp
said at this year developer summit that
the top 10 is a good place to learn to
crawl from
so here we begin
briefly the top 10
because there are full presentations of
the top 10 and heck i even have a couple
myself
are currently the items on the right of
this table
those on the left are from the previous
version which was released three
four years prior to this one was
as you can see the changes are fairly
minor right two got dropped because
framework changes correct the bulk we
previously saw
two got merged
they're actually fairly similar
three got added two of those via the
then newly added community update
created by the absence professionals who
who found that these were important
they are important and at the time data
was hard to find on them
broadly top 10 populates three areas
validation issues access issues build
issues by build i mean
makeup or composition here so education
on the top 10 tends to cover these three
areas to greater or lesser degree using
examples
possibly demonstrations
then talking through code samples and
showing how you can fix it in the code
and how those issues can be uncovered
discovered and fixed remediated
that's exactly what the asc 101 or ase
foundation covers
this is who it's aimed at all of product
delivery from
product owner right the way through
engineers
um you don't need to know anything going
into it it is a foundational it's the
it's the the basic uh
entry level here
um and it'll cover the top 10 and the
basics of appsec
it comes with a slide for each of the
top 10
and speaker notes based off of the
expert experiences the guys who
contributed
and it can be found
right here
just a quick special thank you to hmrc
her majesty's revenues and customs at
the uk tax department who helped build
this piece of work and then contributed
back to oh wasp
taxpayer money paid for this thank you
very much folks
but what comes after that what are the
next steps
copy of course
because architectures are evolutionary
and cornucopia allows developers to use
the knowledge they learn through the
game to evolve the thinking about
threats
through modelling perhaps
as the architecture evolves
we choose cornucopia it's perfect for it
cornucopia itself stands on the giants
that follow
and kind of simplifies those more
complex structures and the top 10 as
well
and it creates a top five
places to consider doing either a user
story or a user security story or a
security spike or adding security nfrs
to
existing stories or acceptance criteria
for security in existing stories
cornucopia in case you don't know
is a card game um and it's based off of
atom showstax eop elevation of privilege
now eop covers game of kai gamification
and gamifying
threat modeling for security folks
well cornucopia does that for developers
the point of view
slips shifts just slightly now i have
hours of presentation and demonstration
materials on cornucopia
and adam and i have done
a panel where we we go back and forth on
on the reds and merits of gamification
and on the different tools we can use
for that
but if you want to learn
how to play the game
then watch this video
and then reach out to me to get a hold
of a deck i have a couple of decks that
i can i can share with the first group
people that reach out
um or for b to access the online version
that we're currently working on
again i have hours on these so i'm going
to just zip through the slides very
quickly
cornucopia or the os top five
is made of six things
you'll see why in a second
those six things are
all then
who you are
auth z
what you're allowed to do
session management the how and how often
we check auth
cryptography which is the secrets the
hashes and
the poor choices
you know what i mean
come on
anyway moving along
data validation encoding which is making
sure that anything that crosses a
security or a trust boundary is safe
and then a catch-all because stuff
doesn't always just work out never fits
nicely
so how do we use cornucopia as an
educative tool
well we play the game
actually as a security person what you
do is you teach developers the game
and give them the materials that come
with the game
and then let them play on their own
otherwise it's a little bit like playing
cards against humanity
with your parents
then to cover off the do threat modeling
check box um you have the devs tag the
stories in epics and jira
and you have them scanning their score
sheet
that gives your auditors something to
feel feel warm and fussy about well you
and the dev teams know that things have
been done right
but how do you level up from that
well there are two possibilities and
this is where most of the work is still
happening
we are to begin with wanting to make
secure software development and skill
that developers can gain while learning
to code
or
learning to code
writes or learning the right coding
patterns
most of us start fairly young
my 11 year old is quite happily building
software
and so we're targeting in this case the
tertiary education
area
with an eye to group skills into modules
and then stack those modules up to cover
the whole of this
builder secure things space
how does it work
basically while the
asps is ospr's answer to the question
what is the standard against which
software can be measured for security
because well it is a standard
and as a standard it has gone through a
lot of rigor and scrutiny though perhaps
never more so than we when we decided it
would make a good basis for the
curriculum
the thousand-foot view of the svs in
case you've never heard of it
the first thing you need to know about
the acs is that it's split into three
levels
each applicable to different types of
applications
level 1 should be used on all apps and
it can be tested for using das tools
alone
zap is a great way to test level 1 osap
take a look
every app should probably get tested
though to at least level two
um the problem is when you go beyond
level one then you need to look at code
you need to review some designs you need
to have conversations to the developers
to get full testing in
level three is for apps that are serious
about security
if your app manages special categories
of data or connects to systems that do
that think bank accounts patient records
etc
then you should be looking here
there's also technically a level zero
which is what any application assessed
that didn't meet all requirements for at
least level one is tagged to spin
if you use level zero in your
organization you probably want to
document what a pass meant for you
learning areas for the svs cover
everything from architecture not which
is not tested during level 1
through to details about configuration
which is incredibly tested during level
1
and includes the domains from cornucopia
earlier right authentication
uh access control authorization
session management validation
cryptography
many others
to assess to level three
across all these domains is to cover off
100 286 requirements
so how do we break this down for
teaching then well
we break all the svs requirements into
actions and terms
we group the terms into units and then
we weigh the actions described
against bloom's taxonomy
bloom's taxonomy or more accurately the
cognitive domain of that taxonomy
describes six levels of learning from
remember through to create
it's a standard applied by educators
around the world and being hierarchical
in order to
successfully understand you also need to
remember
so remember first then understand
in order to apply you need to remember
and then understand and then be able to
apply it
now while six is theoretically possible
we've not find cause to use it in the
afc yet
we've also not seen anything less than a
two
it's interesting to note
the process is as follows we take a
requirement from the svs like this one
we identify the actions
we identify and create the terms and we
apply magic to determine the bloom's
level required to fulfill the
requirement
that magic being
a
fair amount of experience with assessing
against the asvs standard or building
software that met those requirements the
svs standard
this results in the basis the curriculum
though right what knowledge is required
to build something that meets the
requirement or to evaluate whether it
does meet this requirement
obviously the next steps from here are
to find information on each term
make a note to the standard authors
about the item
or maybe even add a knowledge-based
article or some combination of the above
and then actually build teaching
materials from those terms which would
be grouped by units
curriculum may not go that to that
degree but we have done it with the
first section
this particular
requirement actually does have a note
back to the asvs team
and it also has a to-do item a thing
that needs to be done
because we don't have an article on
feature-based approach for modeling
role-based access control
but there is somebody who's written a
paper on it and that's a good source to
start at
now although we covered off asps in the
tertiary section everything gleaned
there can and must also become available
through industry training
as with everything discovered in
industry training should eventually end
up in the tertiary education section
so imagine that all that previous work
would also be broken up into parts that
can be consumed in one three and five
day increments grouped together in
job shaped courses with related modules
rather than the units previously
described
it's all still work in progress
in this section we're going to cover owa
sam
where asps is about building a secure
thing sam is about building a thing
securely
when it comes to telling folks how they
should build software securely
you need it to be prescriptive
which is one of the reasons we picked
sam over for example visa
the other is of course that it's an
oauth project and you may have noticed
how much we like those
sam is made up of five business function
each business function is a category of
activities that organizations involved
in developing software somehow perform
each of those functions has three
security practices
which are group
related security activities that support
them
security practices have six activities
grouped in logical flows divided into
two streams
and split across the three maturity
levels giving us 90 requirements in
total
streams cover different aspects of a
practice and have their own objectives
aligning and linking the activities in
the practice over the different levels
now the learning areas in sam
cover activities from overall governance
and education hey that's a thought
through to operational concerns like
managing incidents
as before with the asps we break down
the activities into actions and terms
and then go through the weighing process
there are fewer activities in sam than
requirements in the svs yay
but the activities are more complex
damn
we follow the process from before
but this time gathering actions
from a given function
uh function slash practice slash stream
for a given maturity level as you see at
the top there
and we group those actions into modules
and then the rest becomes identical
applying this in practice though
how do you get hold of the curriculum
well
foundations
it's easy just go get it
um sc101 is available on the os
website
on the application security curriculum
page that was linked earlier
go grab it teach to your teams
or
have us come in secure delivery a bit of
a pitch here and uh we'll present it to
your teams we leverage other tools like
secure flag which is another os project
and commercial tool and we we make it
live for your devs we give them hands-on
actually break you know break this
application by exporting it using that
vulnerability then show you the
vulnerabilities in code and then help
you to work out how to fix it
intermediate training
use the game with your teams reach out
for a deck of cards
uh again
at root demon
um
or reach out to us for b to access to
the the online version
or have us come and play with your teams
and have us teach them how to play
tertiary and industry training is still
work in progress
we've built parts of the whole and we're
able to present those parts to your
teams if they have a need for particular
aspects of them
but that's not the point right the point
is for this to become a standard and for
everyone to have access to it all of it
so we'll continue to build out what's
there until that is the case
future
well
help us to complete the mappings to work
on building the modules and units and
then possibly even creating slideshows
and speaker notes
sponsor the project
like hmrc has by engaging us
or through the oauth grants process
that's listed here
and most importantly give us your
feedback
that's my erwasp email address there
along with my twitter handle and of
course my discord handle
reach out to me here if you want to as
well
one last thing
there is a survey by the how to get into
abstech project the sister project to
the curriculum
which also falls under the oauth
education committee now they're doing a
survey to get details of abstract roles
around the world
and the survey is going to be used to
create destination personas endpoints in
the journey into appsec
and of course it will provide us with
the basis to build out the job-shaped
courses and those related modules that
we need to do
with that i'd like to thank you very
much for your time and i hope you've
gotten a lot out of today
again feel free to reach out to me any
way you like any of the options that
were available to you even through the
feedback channel here with that i wish
you a fantastic day