hello
welcome to my talk
i'm cheryl biswas i also go by encrypted
and
i really appreciate you taking the time
to discover
software supply chain attacks and the
chinese apts
who have been increasingly behind them
signed sealed and delivered
so
who am i
i work as a threat intel analyst with a
major bank here in toronto canada
i am a founding member of the diana
initiative we just had our this
conference that was online
much thanks and appreciation to
everybody who was able to be part of
this we love supporting inclusion
diversity and just drawing a bigger
circle about who we bring into this
amazing field
i'm a member of the c3x college student
cyber simulation here in toronto and
it's an annual event because it's great
to be able to give college students the
opportunity to learn
in a real life environment
what it feels like to
respond to an incident how to
communicate with management and just to
be able to pay it forward because we all
started somewhere
okay
that's enough exciting things about me i
am a cat mom but you don't get to see my
cats they are in the other room
all right
in this talk i want to explain what
software supply chain attacks are and
the growing threats that they pose
so we're going to take a look at
codependency how that's a factor
and how it's compounded by mistakes and
misconfigurations
we're going to take a look at
some of the attacks and the
state-sponsored actors that are involved
and what this will do is highlight the
prevalence of chinese threat actors who
continue to up their game
and the number of victims
and then we'll wrap up with what we
could do better
and how we need to shift
okay
i thought that this
would be a great starting point
what a year it's been
even before we got out of the game the
tone was set
solar winds was truly unprecedented and
it's the kind of event that we talk
about for decades we build and we learn
off of it
but the thing is
software supply chain attacks aren't
something new
they've been around for many years
and we've been watching that check
engine light for a long time but not
really addressing the issues
so i'm here to tell you there is a lot
more going on than we realize
a supply chain attack
is an abuse of trust
it's compromise
right at the source
this can create access points into the
networks of those customers
into the thousands as was demonstrated
by the attack against solarwinds orion
and as we learned
nobody is untouchable
everyone is a potential target
according to mitre attack a software
supply chain attack happens when hackers
manipulate the code in third-party
software components in order to
compromise the downstream applications
that use them
attackers leverage compromised software
in order to steal data
or to corrupt targeted systems
or to gain access to other parts of a
victim's network through lateral
movement
and any other upstream part of an
organization's supply chain
can then be targeted now that can
include application developers
or publishers of just off-the-shelf
software like solarwinds
but also think of api providers and
there are a lot more apis by the day
and just the open source community in
general
quote the attackers tamper with the
development process of the software to
inject a malicious component
such as a remote access tool
that will allow them to establish a
foothold into the targeted organization
or individual
pretty powerful
very effective
now this is a timeline of events going
from the end of 2020
into this year and as you can see
it got busy
you might even recognize some items on
there and we'll talk a bit about most of
these as we go through
as that attackers will manipulate
software dependencies and development
tools in order to compromise the data or
systems before
before they reach the recipient
and as we've seen
they go after the source code
now in the case of solarwinds
microsoft said that attackers gained
access to some of the source code for
exchange
azure and intune
i don't have to tell you
especially not after this march just how
many exchange boxes are in use out there
or how many many organizations have
azure up and running or in the process
of getting them up and running with the
mass migration to cloud environments
having the source code
lets the attacker look for undiscovered
vulnerabilities in order to exploit them
first
and they like to go after certificates
and that's another thing software
certificates
they're involved in so many of these
attacks pretty much all of them
stolen code signing certificates allow
the attackers to evade detection and to
deliver malware payloads
as though it comes from a legitimate
source because of course with a
certificate why wouldn't it signed
sealed
delivered
this is what
chinese apt barium did
with the azus live update when they
infected asus users on mass
and certificate abuse is an ongoing
component in all kinds of attacks for
example two years ago in an npm attack
developer accounts were targeted
and build
environments were compromised
so let's talk about the who and the why
this is definitely the purview of
state-sponsored threat actors
who have been in many cases identified
as a group of chinese cyber espionage
groups known as abc 10 apt 17 and apt-41
the various other nicknames
as well as some major
russian threat actors
now china has actively targeted tech
companies in taiwan with supply chain
attacks because they see them as major
competition
and this is what where i do threat
intelligence plays a key role because
we're we're watching the geopolitics
play out and we're reaching back for
historical context
on patterns of behavior
because history repeats itself
and actions
have consequences
ian pratt who is the global head of
security for personal systems at hp had
this to say
whether they are a direct target or a
stepping stone to gain access to bigger
targets
as we've seen with the upstream supply
chain attack against solar winds
organizations of all sizes need to be
cognizant of this risk
now both cyber criminal and
state-sponsored groups
target the technology industry because
these companies are relied on by
many organizations and individuals
and that can have a wide-ranging impact
attacks on tech companies can enable
third-party compromise of enterprise
customers
via software supply chain attacks
and that's the beauty of a supply chain
attack
you don't have to go directly after your
target
you come at them sideways
nick weaver the security researcher at
uc berkeley international computer
science institute
shares this
supply chain attacks are scary because
they're really hard to deal with and
because they make it clear that you are
trusting a whole ecology
you are trusting
every vendor whose code is on your
machine and you're trusting every
vendor's vendor
we're grappling with the growing sense
of complexity in tech
and it's not just what corporations
deploy in their own environments
but how they're incorporating
third-party tech
as windows snyder says the product
assumes all the security risk
of all the components that it
incorporates
need to say more
as i mentioned earlier
we
are increasingly code dependent
sorry bad pun
the fact is that
applications increasingly depend on
external software to work
there's proprietary code
open source components
third-party apis
modern apps are simply too big for just
one developer to try and do on their own
so software reuse
has become the norm
and there's popular open source projects
which are used as dependencies
these become attractive targets for an
attacker
who can then add malicious code to them
and claim the users of those
dependencies
for github security researcher maya
kawazurowski
85 to 97 of enterprise software code
bases
come from open source components
yeah that is a big number
what's the average project having
203 dependencies
that's a lot of trust involved
now given how sophisticated these
attacks can get
any project that doesn't incorporate
basic protections
like code signing puts itself at
considerable risk
again
we're using other people's technology
this is about trust issues
this is a timeline
of software package repositories that
have been involved in supply chain
attacks
it's really easy to just get code from
various projects online and then
incorporate that into other software
but here's the risk
some of those open source projects are
widely used but they're not well
maintained some have even been abandoned
code reuse can help to simplify and
speed up application development
but it's at the cost of being vulnerable
to
compromised off-the-shelf components
now for an attacker compromising a
software supply chain can be through the
manipulation of the application source
code
or the manipulation of the update and
distribution methods
or by replacing compiled releases with
modified versions
and targets can range from either a
really specific and limited group
to a very
wide range
and that brings us here
the continuous integration and
continuous delivery pipeline ci cd
it's considered a best practice for
devops
because it helps them to deliver code
changes frequently and reliably it's a
good thing
now ci is a coding philosophy it's a set
of practices to help drive development
teams
to implement small changes
and then check in code to version
control repositories frequently again
this is a great practice
it's a consistent automated way to build
package
and test applications
with one caveat
we're using other people's technology
mistakes will be made
yes
in mcafee's blog from
last july
sikar sarokai and i hope i didn't mangle
that
had this to talk about with regard to a
source code leak
and
what we should have learned from that
and how we need to protect our ips
there was a sonar cube misconfiguration
and it led to a massive leak of source
code which affected 50 major companies
sonarcube is an open source tool that's
used for static code analysis and to
check for bugs before deployment
now based on the attacks
developers and insecure development
pipelines are ideal targets either for
state-backed
apts or
highly resourced criminal groups
and let's discover what those
adversarial inclinations
look like
this is
from a report called broken trust and it
was released by the atlantic council
earlier this year following
solar winds
there was a lot to be processed
there have been 36 other cases of
intruders successfully targeting
software updates from 138 recorded
supply chain attacks and vulnerability
disclosures
now of these 36
15 had similar access to build or update
the infrastructure
think about that that is a lot of
control when you can tamper with and
modify existing infrastructure
and half of those 15 could be attributed
to nation states
on december the 14th 2020
the news hit
now solarwinds orion is trusted network
management and monitoring software it's
used by governments
fortune 500s
security companies like
fireeye
major tech companies like microsoft
and nobody knew anything was wrong
what's distinctive here
is the degree of stealth
the ability of the adversary to conceal
their actions
and the length of time it took
for this to be discovered this was an
operation that took time and precision
to do the necessary reconnaissance
the sophistication to tailor all the
pieces
and then the patience to just let it
play out and avoid detection
they targeted and compromised the
software build environment and code
signing infrastructure for orion
code there's that word
code signing
they modified the source code
to add a backdoor
and they signed it
they leveraged the existing software
release management system
and they used stolen certificate
certificates to laterally move through
chains of trust
signed
sealed
delivered
this attack
is significant
and i want to tell you a bit about it
because it's something we all need to
learn from
and remember that things continue to
evolve and this just presents
opportunity to the attacker
security researcher alex berson
shot a bright light on a scary
possibility
he took a hypothesis
about a supply chain substitution attack
where a software installer script is
tricked into pulling malicious code
files from a public repository
instead of getting the intended files
with the same name
from an internal repository
and then he targeted apple
microsoft tesla and about 32 other
companies to execute unauthorized code
inside their networks to prove that this
would work
it did
however
somebody's watching somebody's always
watching
and in this case that somebody thought
that it would be a good idea too and
without let's say that permission they
targeted some other companies in march
of this year
again microsoft but also amazon slack
lyft and zillow to name a few
this premise prompted a study by
researchers at the red hunt labs
what did they find
well
93 repositories out of the top 1000
github organizations
are using a package that doesn't exist
on a public package index
this can be claimed by an attacker to
cause a supply chain attack
and 169 repositories were found to be
installing dependencies from a host that
isn't reachable over the internet
126 repositories were installing
packages owned by a github or get lab
user
that doesn't exist
now of the top thousand organizations
that were scanned
212 had at least one dependency
confusion related misconfiguration in
their code base
this is significant
because according to the researchers
much of the open source ecosystem
depends on these giants
we know that these repositories have a
lot of users so it stands to reason that
if any of their projects were to get
affected
there is significant likelihood
that millions of users could be at risk
xcode spy
so
we know that attackers are targeting
developers and they're targeting the
shared sites the repositories where code
is uploaded for use by others
in march this year a new malware variant
was observed targeting ios developers in
a supply chain attack to install a back
door on the developer's computer
so xcode spy is a malicious product
project
and it affects the free application
development environment
it's common behavior by developers to
share their projects online with other
users
it's just good sense
it's collaborative and it's efficient
well threat actors
behind this attack
abuse
this norm and they used a legitimate
development environment created by apple
newest
to fool victims into adding an online
project to their application that would
compromise their system in
a supply chain attack
codecov
is an online platform used for hosting
code testing reports and statistics
now it provides developers with tools to
help them quantify just how much source
code gets executed during testing
and they serve over 29 000 customers
globally many of these are enterprise
level clients like godaddy atlassian
royal bank of canada
procter gamble
but the impact went beyond this
to thousands of public development
projects i think kubernetes
pi test
ansible
victims included twilio
and rapid7
and the e-commerce platform makari
on april 1st
of this year kodkov reported a supply
chain attack that had occurred back in
late january
what happened was this
attackers leveraged an error in the
process that creates codecov's docker
image now this allowed for them to
extract credentials
which protect the modification of the
bash uploader script
this is a tool used by customers to send
code coverage reports to the platform
the script was modified
to deliver details from customer
environments to a server outside of
codecov attackers could export
credentials tokens or keys which pass
through codecov's continuous integration
environment which we've talked about
they could then use these
to access services data stores or
application code
in 2009
operation aurora served as a wake-up
call when the chinese state-sponsored
group ept 17 targeted google adobe and
some other tech firms for their source
code management systems
in order to alter that source code
we're going to talk a bit more about
that later
in
2017 there was not petcha
the elite russian hacking team sandworm
who's part of the gru military
intelligence service
compromised and took over the software
updates of medoc which is accounting
software used throughout the ukraine
and they used this to distribute
destructive malware known as not pecha
this infected major companies like
maersk
as unintended consequences
and the costs amounted to approximately
10 billion globally
in january of 2019
there was a sophisticated supply chain
attack
that targeted the asus live update
utility and that's something that's
pre-installed on pretty much
all asus computers
and an auto update feature and we'll
cover that a little bit later as well
this group also delivered something
known as shadowpad malware to infect
enterprise networks
using a product known as netserrang
nectaring specializes in
server management
and security connectivity software
if this sounds familiar
it should
i think that just happened to solarwinds
as promised let's take a look at the
chinese apt groups who've been involved
in software supply chain attacks
now all state-sponsored adversaries
chinese cyber espionage groups have been
and will continue to be the biggest
threat to tech
conducting economic espionage and
intellectual property theft
technology companies are rich targets
just on their own
but these groups leverage them
to infect supply chains
to go after their customers
and they are the leaders of the pack in
terms of the number of attacks and their
capabilities
these are some
of the most well-known attacks of
chinese state-sponsored threat actors
so i'll go through them briefly
and highlight where attribution was able
to be made some key lessons about the
targeting intrusion
and just the issue of trust
something that really came across to me
while i was doing the research on this
was the overlap in terms of tactics and
tools
my personal observation would be that
these groups tend to work closely
together
and they probably were learning within
one group
then moving on to another and then they
took what they had developed and learned
to leverage it as needed because that's
efficient
and it's collaborative
and isn't that what we do when we're
talking about open source development
exactly
why reinvent the wheel when you already
have something perfectly useful
when somebody's already covered that
ground
these groups are united though
in how they're serving their state and
its mandate
now if you
wanted to try mapping chinese apts to
government and military
i found this work that was already in
progress it's by anastasius pingios and
i'll share the link on the next slide
it's very complex but this is a great
visual representation
i'll give you a moment to take a look at
that so
you can capture it
and if you're at all familiar with the
crowdstrike they like to uh to group
their
adversaries in
fun and colorful ways
all right so let's start with 2009 in
operation aurora
like i said earlier it really was a
wake-up call
apt 17 had targeted google adobe and
other tech companies
to go after their source code by
tampering with their source quote
management systems
this led to google implementing zero
trust
to track lateral movement and to
implement better infrastructure
those were their lessons learned
and what's interesting about this is
that
in the fallout of solarwinds orion
google wasn't one of the companies
mentioned
in 2017 an attack was discovered that
targeted an administrative software
package known as evlog
this was made by altair technologies who
are a canadian software company
this was a software supply chain
intrusion
which targeted enterprise organizations
globally including military
organizations and defense contractors
which
is something china cares deeply about
developing
and surpassing the west ad
there were also banks and universities
and there were several major telecom
providers
i get a little worried by targeted
attacks against telcos for a few reasons
and that's not just because akamai had a
hiccup and went down but
we all feel the pain
when we lose our online access for any
degree of time
it's
pretty much because we can't live
without it
now in this attack
there was a very impressive client list
and you should be thinking about
solarwinds because that too was a very
impressive quiet list
several years later
we still don't know how many of these
customers
were
and could still be
compromised
now
why evlogs
because the users were mostly system and
domain admins
and that offered excellent access to
targeted networks after the initial
compromise
reconnaissance is really important
i think so many people may have already
heard about this one
but ccleaner
this has been linked to abt 17
and more specifically to a subgroup
known as axiom who have historically
been engaged in supply chain attacks
the c clean air attack showcases
technical knowledge preparation
and patience
the timing was strategically
advantageous because the original owner
puriform was in the process of selling
to avast
that creates a lot of distraction and
confusion
so it's really easy to miss things like
things in the network that don't belong
there
the attackers took their time to move
laterally in the network
during off hours to avoid detection
a month they installed a modified
version of shadowpad backdoor malware
to escalate their privileges
then
they distributed a cryptographically
signed version
of the a modified ccleaner product
and no one suspected sign sealed
delivered
if we want to look at this a little more
technically
and just walk through
the actual uh steps of the attack
initial compromise came through
unattended
workstations from a cleaner developer
they were connected to the piriform
network
and the attackers utilized
teamviewer
they also reused credentials that they
found from previous data breaches in
order to access that teamviewer account
they delivered the malware using
vbscript
and
they developed a malicious version of
ccleaner
they used rdp
to open the back door on a second
unattended but connected computer
there's some good lessons in here if
you're taking notes
there they dropped the binary and a
malicious payload of second stage
malware and that was delivered to 40
ccleaner users
now they compiled a customized version
of the shadow pad back door
in order
to allow for further malicious downloads
and data theft
in preparation for a third stage
and then they installed their third
stage payload
now that malicious version
had
multi-stage malware in order to steal
the data and send it back to the cnc
and
as far as action on objectives
very likely that stolen data has been
put to great use in further espionage
activities
if you wanted to try and go through this
against miters framework
there's a there's a lot
i think that would be applicable for
example
under recon you've got
a lot of work that they did
but supply chain compromise
resource development
they compromised accounts
i believe they compromised the
infrastructure they certainly developed
capabilities and they established
accounts
for their initial access
thanks to those
reused credentials
they found themselves a valid account to
get in with
for execution they leveraged some
software deployment tools
and for persistence well
there was more account discovery and
account manipulation to keep them in
for privilege escalation
exploitation for privilege escalation
for credential access
they had credentials i believe from
password stores
discovery
can i say they discovered everything
that there was to discover
lateral movement
as we talked about they used shadow pad
and remote surfaces
and then exfiltration over a c2 channel
and the impact being some data
manipulation
and exfiltration
shadow
hammer this is the asus
attack in january 2019 a sophisticated
supply chain attack was discovered
targeting the asus live update utility
so this is something that's
pre-installed on most aces computers
and it's to make life easier for the end
user it automatically updates components
like the bios
uefi drivers and applications
according to gartner in 2017 asus was
the world's fifth
largest pc vendor which would make it an
extremely attractive target for apt
groups that might want to take advantage
of their user base
and in this case that apt was chinese
threat group apt-17 or barium
who were as we've seen behind the attack
on c-cleaner
the attackers altered an older version
of the azus live update utility software
and then they distributed
their modified version to the azus
computers around the world
the software looked legitimate
it was signed with legitimate asus tech
certificates
and it was stored on official servers
and it was even
the same file size
signed
sealed delivered
and once it was planted
that backdoor program gave attackers
control of the target computers through
remote servers that let them install
additional malware
in 2020
there was an attack known as able
desktop
by the apt group lucky mouse
it involved tool reuse by other actors
that were not only chinese groups
and i thought that this was interesting
because the shared use of tools and
collaboration
within chinese state-sponsored groups
affects their ability
to do
accurate attribution
they compromise the able desktop chat
software that's used by mongolian
government
agencies and then they hijacked updates
of the software supply chain
this is a really great small picture
of what could be a much larger more
devastating attack
and again
the attackers didn't need to steal or
forge an update signature
because abel's updates were not signed
another attack coming out of 2020 was
sign site
this is the vietnamese government
certificate authority and it was the
target of a software supply chain
intrusion which targeted a wide range of
public and private entities
the attackers used its digital signature
software
that provides certificates of validation
and software suites
to handle digital document signatures
now this software
is widely used throughout vietnam and
it's mandated in some cases
and if you've seen if you think of not
pecha
and how that tax software was mandatory
and there are other cases this is what
attackers for looking through their lens
are leveraging against us this is truly
an abusive trust
so the key point
the abuse of trust here leveraging a
service oligopoly
we haven't got actual confirmation of
who was behind it but it's definitely
believed to be a chinese state-sponsored
group
possibly a group known as ta-428
who have a track record of targeting
east asian countries like mongolia
vietnam and have possibly even gone into
russia for intel gathering
and that brings us to golden spy
golden spy malware was embedded in
required tax payment software
issued to corporations who wanted to
conduct business operations in china
now chinese banks require businesses to
install
an intelligent tax
to pay the local taxes this this
intelligence tax software and it's
produced by the golden tax department of
the icino corporation so the malware
install the back door on systems which
enable a remote thread actor to execute
windows commands
or upload and execute any binary that
could be ransomware remote access
trojans anything
and the malware provided system level
privileges
so the capability to execute any command
or any software on that system where
it's installed
and it was connected to a cnc that was
distinct
and separate from the tax software
network infrastructure
you've probably heard of these guys
this year there was a very
very very large breach and it's just
coming to light the details around it
ceta is a global i.t provider for
90 of the world's airline industry
the pieces of this when you put them
together
link back
to show that chinese state-sponsored
threat after apt-41 was involved
and it's potentially impacted over four
and a half million passengers
ceta had announced the attack back in
march
and then soon after singapore and
malaysia airlines
disclosed that their customers personal
data had been exposed
after that
air india reported a major attack
against its systems
now
group ib took a closer look at this
they they didn't believe what they saw
at first and then they realized just
the level of sophistication involved
in what was an enormous level of attack
that could only be attributed to a
state-sponsored actor
and that's how with some digging they
were able to find artifacts through the
malware through overlap in terms of
the code that they were seeing to trace
it back to apt-41
apt-41 also known as wicked spider went
t with ties to barium
a group that's been active since 2007
and
has a track record in supply chain
attacks
and
that's that's why it's so hard my
attribution is so hard
all right
so let's talk about
what we should and could learn
from these attacks
we do
have a new executive order
and that's a great step in the right
direction
but the picture is far bigger than we
realize
these are some recommendations
we need to have prompt communication
when something happens
we need to be able to share information
effectively
and we have to take action
because time is a luxury we don't have
and something that our adversaries have
plenty of
code signing
is essential
even though as we've seen certificate
abuse can be rampant
we need to find ways to ensure that tech
is secured by default
establish an international
norm with clear penalties because you
can't have something without the other
if you're going to have policies you
need to have enforcement
eric sheehan of symantec who was also
integral in helping investigate and
decode stuxnet
says that this is like finding a needle
in a haystack to have the right security
telemetry and visibility at the right
control points
in your organization
and here's the shift
and so
i'd like to end with this
i love podcasts and one of my go-to's is
risky business
they had an excellent segment about
operation aurora
with mark rogers from octa whom i've
referenced a few times in here
so
if you can gain a position of trust
you can exploit it
think service or non-person ids
accounts that nobody really scrutinizes
with enough privilege to leverage to do
e-discovery
and to find all the stuff that you need
for an attack
your chain is only as good as its
weakest link
and there are more ways to abuse the
chain of trust
than people realize
thank you so very much
i hope that you got some good stuff to
work with out of this talk and i really
appreciate your time my details are
there
you can find me on twitter
thank you