my name is sebastian bay and i'm a
researcher with the swedish defense
research agency specializing in election
security and digital harms
this year the defcon voting village is
focused on the spread of election
related disinformation
and misinformation and i'm going to give
a short talk introducing some of my
research
and highlighting the need for increased
cyber security
for social media companies as part of
our combined
efforts to strengthen election security
there are of course several aspects of
election security
and traditionally we have focused on the
conduct of elections when we talk about
cyber security
that means safeguarding the i.t systems
needed to administer elections from
registering voters
to counting and tallying the results
this is clear
and this is also something that is
needed we have seen
continuous effort to try to undermine
election systems using i.t
attacks of different sorts however
recently we've also seen that digital
disinformation
is an equal or perhaps even larger
threat to the conduct of elections
where digital disinformation is used to
try to undermine
trust in the election process it is used
to try to undermine
the will and the ability of people to
participate in the elections
and it is of course used in an attempt
to toward or undermine the political
process
often with the illegitimate use of
influence in different forms
but i will argue today that much or
a huge part of this problem is also a
cyber security problem
but a cyber security problem primarily
for social media companies
now if we simplify enough this
information
can be um divided into two separate
issues
the issue of content and the issue of
inauthentic behavior
now content can be disinformation it can
be illegal
and it can be many other things and
we've seen during the last few years
that regulating and moderating content
is difficult
even if we're now seeing better policies
in regard to election related content
on social media platforms but if
content is tricky to develop clear
policies for
inauthentic behavior and other forms of
social media manipulation
is far less tricky it is simply not
allowed
on all the major uh platforms i've
studied
you are simply not allowed to run
thousands of fake accounts
on any of the large platforms you're not
allowed to manipulate their algorithms
you are not allowed to scrape their
content you are not allowed to hack
their systems
yet we see that this
is a large problem it is a large
industry and of course this is something
that is being done
all the time this is what a bot farm can
look like
in real life these pictures are provided
by the ukrainian security services
showing what they argue is a
russian-sponsored bot farm
inside of ukraine to try to avoid
detection
what we're seeing is sim card it is
various forms of antennas used to run
used to run different forms of proxies
and of course to set up and avoid
detection
now the european union has long
underscored a need
for social media companies to intensify
and demonstrate the effectiveness of
efforts to close
fake accounts this has been in a code of
practice
to address the spread of this
information also within an election
related context
the social media companies have been
asked by the european commission
to conduct regular reporting to the
commission
where they report on the number of fake
accounts that have been closed
during the last quarter and during the
last year and we've seen in this
reporting
that fake accounts continue to be a huge
problem
measured in the billions and it is a
large problem
because fake engagement trigger
algorithms to spread
content to authentic users fake
engagement
trick authentic users to believe that
content is more popular than it really
is
fake engagement misleads users
fake engagement manipulate democratic
conversation
and fake engagement create loss of
genuine
advertisement spend
i've co-authored three reports on the
topic of
inauthentic behavior on social media
ranging from the black market
to trying to develop metrics and methods
for assessing the ability
of social media companies to counter the
abuse
of their platforms in effect trying to
assess
the level of cybersecurity in these
platforms when it comes to
preventing inauthentic behavior
if we start by looking into the black
market for social media manipulation
and which i think is a good base for
understanding the problem of social
media manipulation
i'm going to give you three main
takeaways here
the first is that the scale of the black
market infrastructure
is extensive we see that an
entire industry has developed not only
around providing the manipulation
services
but providing the infrastructure needed
for the manipulation services
to work and that ranges from
fake sim cards digital fingerprinting
scripts and capture services
for the manipulation services to provide
that is used to generate provide and
maintain fake accounts and we also see
management platforms that are used by
the
inauthentic engagement services
but they're also sold as software to um
to independent contractors and to
private companies that wish to run their
own campaigns
seeing this entire service we initially
thought that this was a black market
and that is also why we labeled the
report a black market for social media
manipulation
but what we saw is that it's not
actually a black market
it is an illegitimate market perhaps but
it's
extremely easy to find and the openness
of this industry
is still today quite striking
we see that the larger social media
manipulation service providers they
fearlessly promote their services they
promote them on their own websites of
course
but they also promote them um on app
stores
on um on the social media platforms
themselves uh they usually run tutorial
accounts on
youtube facebook instagram etc
and of course you can find them in all
major searches
it is even so that they they use
ads on search engines to promote their
services
third much of this infrastructure seems
to be russian
it doesn't mean that it's
state-sponsored but many of these
primary service providers that are
reselling their services to many of the
companies
offering manipulation services in the
west
are russian and they seem to be russian
simply because
this is a place where this industry has
existed for a long time
and there is a lot of know-how in this
area
but this is also a global industry
and we're seeing more and more of
fragmentation uh where for an example
uh if you live in nigeria and you want
to buy manipulation you might go to a
nigerian provider
they might use russian infrastructure
and
but they might also use infrastructure
from southeast asia etc
and we're seeing these companies in all
parts of the world
even though a lot of the services
provided are being generated in
southeast asia or russia
we're seeing resellers and customer
support and
development in the west as well
so social media manipulation we've tried
to assess
to what extent social media companies
are able to counter the manipulation of
their services
how good they are and how big of a
difference it is between the individual
companies and we've done this in two
consecutive reports in 2019 and 2020
and we're now in the process of setting
up an experiment for 2021
and this of course is done to assess
the ability of social media companies to
combat inauthentic behavior
on their platforms in 2019 we ran an
experiment
where we bought engagement during two
months
on 105 different posts on facebook
instagram twitter and youtube in 2019 we
bought
54 000 fake engagement from these
social media manipulation service
providers
and 3 000 comments more than 20 000
likes
and more than 20 000 views for 300 euros
the equivalent in dollars and we ran
re-ran this experiment during six weeks
in october november 2020
in the context of the u.s election and
we then bought engagement on 39
different posts also including tick tock
last year we increased the number of
engagements bought so we bought
335 000 engagement still spending
roughly 300 euros
after buying this fake engagements we
measured
how much of it got through how much of
it got blocked
and when what happened when we reported
it to the social media companies
our main takeaway was that social media
companies overall
were unable to block the inauthentic
engagement bot
four weeks after purchase more than 98
of the bot
engagement were still online four days
after reporting a sample of the
accounts inaudible were still active
our conclusion last year was that
facebook instagram twitter
youtube and tick tock are still failing
to sufficiently combat
inauthentic behavior on their platforms
enabling the widespread disinformation
and a dissemination of this information
on their platforms
we also looked at several of the
parameters and
that are very useful for understanding
the scope and scale
of this problem for an example we've
tracked the cost of manipulating social
media platforms
from 2018 up until 2020 by
creating baskets that
contain like likes comments
retweets etc and we've seen that a
standardized basket
from 2018 to 2019 and
got cheaper and then from 2019 to 2020
the prices
in general leveled off but we can also
see here that there's a difference
between the
various social media platforms that it
differs between the companies
how the prices have changed and why is
the price
interesting well we believe the prices
is an
indicator for how difficult it is to
manipulate
platforms the stronger security the
platforms have
the more difficult it will be to
manipulate them the more it will cost
the manipulation service providers
and we're seeing that there isn't a
fundamental change in the price
but we're also seeing that there's a
difference between the platforms
regarding how difficult they are
to manipulate we've also looked at the
speed of delivery
that is how quick or how fast you can
manipulate
social media platforms
and if you take 2020 for an example
we could see that for an example tick
tock
last year um was not sufficiently
effective when it came to countering
inauthentic bot behavior almost all of
it were delivered
instantaneously on tick tock while for
example some
of the other platforms about half of the
content were
delivered within 12 hours and then it
took um
in many cases several days before 100 of
the bot
engagement were delivered and of course
the speed of delivery is an important
indicator
for how effective the platforms are at
countering abuse of their services
another indicator is the ability of the
platforms
to remove fake accounts reported to them
and last year when we tested this um
several of the platforms didn't remove
any of the reported accounts
the most effective platform facebook uh
only
removed nine percent or nine out of the
100
fake accounts that we reported to them
overall our takeaway from these
experiments have been that there is a
significant
difference between the different social
media platforms
the amount of money the amount of
resources and the amount of
human skill spent to directing and
trying to combat
platform manipulation makes a huge
difference
and we can see that when platforms
make a concerted effort to try to change
this
we also see that it becomes harder to
manipulate the platforms
facebook has made progress during the
last year for an example
they've become much better even though
we assessed that
twitter is still the industry leader
when it comes to countering abuse of
their systems
and tick tock which was a new platform
last year
scored at least well
but we have reason to believe uh that
they have improved during this year
uh spending more efforts more resources
on trying to improve themselves
uh it will be interesting to see during
the rerun of this experiment
uh this year if this ranking stays
or if we can see that um that some of
these platforms which we suspect have
put more effort into
combating this problem also show a
significant improvement
in our measurements we have seen
uh another interesting takeaway that
we've seen is that there can even be a
significant difference
between um platforms owned by the same
company
and the best example is the significant
difference between facebook
and instagram instagram is much more
easy
to manipulate than facebook is and this
is surprising
one would think that um two platforms
owned by the same company
would have equal levels of security but
this isn't the case at all
um from creating fake accounts
to buying fake engagement there's a
clear difference between these two
platforms
and that illustrates that this is a
technical problem to a large extent
um when we ran this experiment two years
ago we even saw
on instagram that uh even when fake
accounts
that had delivered fake engagements were
removed
the fake engagement remained which of
course is a
technical issue with the platforms so
to a large extent combating manipulation
of
social media platforms is a technical
problem
um that social media companies have to
pour
financial resources into solving and
that hasn't been done
enough before but we have seen
from year to year from 2019 to 2020
improvement
but as it was at the end of last year
when we ran this experiment
the manipulation service providers were
still winning by a large margin
that is you could still effectively and
cheaply buy manipulation
and it would be speedily delivered onto
social media platforms and it would
remain
up for weeks and weeks um
to this day some of this fake
engagements that we bought
already back in 2019 remain active
online
so seeing this and studying the problem
of this information and misinformation
on social media we understand that
inauthentic behavior is a central
component
of coordinated inauthentic behavior
that is the coordinated spread of
disinformation
and one important solution for tackling
that
is through enhanced cyber security for
social media companies
protecting their platforms against
technical abuse
so in that sense social media cyber
security
equals election security because we're
seeing that the spread of
this information we're spread seeing
that
that intentional efforts to undermine
the will and ability of voters
to vote on election day and we're seeing
that the
intentional manipulation of political
conversations on social media platforms
um happen online and some of it also
happens using
bots and technical manipulation
and that can easily be prevented
with additional cyber security for the
social media companies
[Music]
during last year's experiment we
developed a number of recommendations
for platforms and for policy makers
um first of all of course the social
media platforms need to do more to
counter abuse of their services
but also that we need to set standards
and require
reporting from the companies based on
more meaningful criteria
today it's very difficult to compare and
contrast
reports from different social media
companies
we also need to increase transparency
and enable
independent verification of figures
reported by the social media companies
and in the same way we need independent
and well-resourced oversight
we also need to regulate the market for
social media manipulation
we need to counter manipulation services
to a much larger extent
we have seen some important steps being
taken by social media companies suing
manipulation service providers and we've
also seen a number of governments
file charges but still today most of
these
core services remain online
and some more needs to be done but we
also need to understand
that we need a whole of industry
solution to combat this problem
and from a cyber security standpoint
if we take the case of the ukrainian
security services
we could see that these bot farms they
heavily rely on
sim cards and other telecommunication
equipment
and that of course means that regulation
of this
equipment of sim cards remains a
critical component
in combating the misuse of social media
platforms
but we also need to to make sure that
social media companies spend more
efforts
preventing the abuse of their services
so cyber security
remains and will continue to be for the
coming years
a very critical component when it comes
to election security
especially in the field of
disinformation and misinformation
thank you