and welcome to our bar codes on balance
bad
my name is kevin skoglund i'm a voting
system security researcher who's studied
barcodes for several years and reported
vulnerabilities i've found
today i'm going to discuss how voting
systems store votes and barcodes explain
how to decode them
and explore their attack surface from a
security perspective
there are many places where barcodes may
be used in elections
you may see them in the margins of
ballots where they tell the tabulator
how to map ovals to a set of contests
and candidates
you may see them on mail-in ballots poll
books or to keep track of equipment
these barcodes may have vulnerabilities
maybe even ones that we'll discuss
but we aren't going to focus on them we
will focus on barcodes on ballots
that store vote selections
in 2004 the populex slate was the first
ballot marking device to store vote
selections
inside a barcode but it was not widely
used
barcodes on ballots really took off a
decade later with the introduction of
the esns
express vote it was quickly adopted by
many states
like many ballot marking devices it was
used by voters with disabilities
but esns also marketed it to a wider
audience for use in early voting centers
and for universal use where every voter
in a precinct votes on a touch screen
you can see in the photo that the ballot
has a section of barcodes at the top
and a list of the selected candidates
printed below
since then similar products have been
introduced by other vendors
this category of voting systems is still
new but it's spread quickly
they are used by several whole states
and a number of populous cities
there are many pros and cons to using
these devices but i want to spend our
time on the security issues around the
bar codes specifically
for anyone who's not familiar with
ballot marking devices it may help to
have a quick overview of how they're
used
a voter would use a touch screen
computer known as a ballot marking
device or bmd for short
to make vote selections the voter prints
those selections on a printer
the printer outputs a paper ballot which
contains barcodes and a text summary
of the selections the voter has an
opportunity to examine the text summary
and then cast the ballot by feeding it
into a tabulator
the tabulator reads the barcodes and
attributes votes to candidates
then the ballot is impounded into the
ballot box there are variations
but this is the general process now that
we have some background
let's learn how to decode the barcodes
on a few different systems
we will look at the esns express vote in
expressvote xl
the dominion image cast x and the unison
freedom vote tablet
i picked these as popular examples that
illustrate three different ways barcodes
are used
they make good case studies and they can
act as stand-ins for other systems that
use similar concepts
maybe even systems that are not yet on
the market
the sns expressvote ballot uses bar
codes in the code
128 c format the long
code at the top contains the ballot
style which sets which contests and
candidates are being voted on
it also contains a count of the number
of barcodes that follow in the three
columns below it
those are the vote selections this
ballot has 15 barcodes
in five rows if you look at the text
below the barcodes
you'll find that there are 16 contests
one contest had no selection
so no barcode is stored for it
the barcodes can be scanned by any
barcode reader however
smartphones have difficulty because
there's so many barcodes and they're so
close together that it can't pick out
just one
there are other obstacles to decoding
these barcodes
for one thing the barcodes are not
printed in the same order as the contest
in the text for example the sixth
barcode
is a vote for the second candidate in
the summary that makes it difficult to
match them up and account for them all
and the barcodes do not decode to text
they contain six digit numbers
like 093411
that number is a candidate identifier
the tabulator which processes the
barcode
will match that identifier to a
candidate's tally and record a vote
the candidate identifier also matches
the ovals on a marked sense ballot
which you may know is a full ballot or a
handmarked paper ballot
i'm going to call it a mark sense ballot
because the scanner senses the marks
that are in the ovals
with that ballot in hand we can work out
the meaning behind the identifier
the timing marks around the outside edge
create a grid of coordinates
and we can locate the ovals by those
coordinates
the six digits follow a pattern two
digits for column number
two digits for row number one digit for
page number
and one digit for side number and so on
the grid
09 1011 would be the ninth column
the tenth row of page one side one which
is the candidate hope s
lost two positions lower on the grid
is candidate id o nine one two one one
which is rufus lee king
below that is a write in at 09 1411
write-in votes would store the number
only not the candidate's name
the barcodes are strictly six-digit
numbers
overall it's a clever way to allow the
tabulator
to scan two very different types of
ballots but only work with one set of
identifiers
scanning a mark since ballot returns
grid positions and scanning barcodes
also returns grid positions
on the dominion imagecast x palette the
barcode is in the qr code format
the bmd version prints on an eight and a
half by eleven sheet of paper
the vvpap version prints on a narrow
spool of paper inside a box
the qr code includes the ballot style
like the long barcode did on the esns
ballot
it contains the vote selections it
encodes any write-in candidate names
and it has an hmac signature for long
ballots it can print more than one qr
code if it needs to
most barcode scanners and smartphones
will not read these barcodes
most qr codes contain an identifier
that's for an application to use
such as for package tracking or they may
contain a url
you may have used one in a restaurant in
order to look up the menu
these qr codes are not standard because
they contain binary data with a
different structure
instead of just characters if you want
to decode them
you can use a couple of websites or a
unix program called
zbar image any reader needs to extract
raw bytes either as binary or
hexadecimal
in order to work with them if we take a
dominion ballot barcode
and we run it through z bar image it
will return bytes of data as hexadecimal
numbers
every two hex digits is one byte or
eight bits
the spaces separate those bytes
to understand the vote selections we
need the binary so i used some ruby code
to convert the hex digits to binary two
at a time
i colored the first four bytes
differently so you can see how the two
versions line up
it's really just condensing and
expanding the same information
now that we have binary data what does
it mean
we can parse it into several sections
the green parts
are metadata many of them are just
separators
the blue parts are the precinct and
ballot style
if you convert them to decimal numbers
you might recognize them
the red part contains the vote
selections
and the gray part is a 32 byte hmac
which is a hash of the selections data
you can think of the hmac like a
fingerprint for this set of selections
it's not a fingerprint for this
particular ballot but any ballot with
the same choices
would have the same fingerprint
let's focus on the vote selections as
with the esns ballot
we need a marked sense ballot or similar
reference to make sense of this encoding
if we made a list of the contests and
then a list of the candidates in each
contest
in order it would line up perfectly with
the list of zeros and ones
i just took away the spaces and added
line returns
each one indicates a vote each zero
represents no vote
for example the court clerk contest has
three candidates
the binary tells us that the first one
has a vote and the second two do not
this is a straightforward example they
get more complicated with ballot sheets
and write-ins and rank-choice voting
but the basic pattern is the same
let's look at our last example the
ballot from the unison freedom boat
tablet
it has a non-standard barcode at the
very bottom you see a barcode that's in
the code 128 format
which contains the ballot style above
that
the vote selections are marked in a grid
pattern
this is similar to the punch cards of
the past or to a system called
inca vote which is how i refer to these
you can also think of them as compressed
vote targets
vote targets are what we call the ovals
or boxes on a full ballot
imagine removing all the text from a
full ballot and then compressing the
remaining boxes down
to a small bit of real estate to
illustrate
here's a unison mark sense ballot on the
left alongside the grid of selections
and the text summary the outermost marks
in the grid
are timing marks just the same as the
timing marks around the full ballot
in both cases they help the scanner
locate the vote targets
the vote selections are inside the
orange box
there are four of them the text summary
lists five contests
but the governor's contest had no
selection
it helps a lot if i add boxes to the
grid to show you the vote choices that
were not marked
the votes are recorded starting in the
upper left and moving down
each column the first box is empty it
corresponds to the straight party
republican vote on the full ballot
the second box is filled we can look at
the full ballot and see that this is a
vote for the straight party democratic
choice
and that's what the text summary shows
too we move to the next column
the box is filled on the full ballot
that would be a vote for daniel boone
which is also what the summary shows and
so on
notice that the text summary also
includes position numbers
that match the positions in the barcode
so is this even a barcode well
technically probably not
but it is one for our purposes it
encodes the vote selections
in a machine readable format that voters
cannot read or verify
and it can be attacked we'll get to see
how a custom roll-your-own barcode
stands up to those attacks
here's the table that compares the key
features of these three examples
none of the barcodes that we examined
are readable by a voter with a
smartphone in the voting booth
and even if they have a fancier barcode
scanner all of them need a full ballot
to use as a reference when decoding
none of them are encrypted and they vary
widely in their use of integrity
protection measures
now let's talk about potential attacks
on barcodes and on barcode scanners
in this section i'm not claiming
vulnerabilities in any specific systems
i will point out characteristics of some
systems to illustrate how issues might
be handled
i'm not examining the feasibility or the
likelihood of exploitation which are
very important considerations
for one thing i'm not looking at what
other defenses may exist
and i'm not planning to give enough
information to perform an attack
this will be an examination of the
attack surface and the areas that have
potential risks
you can think of it as advice that i'd
give to a pen tester who's been tasked
with examining a system that uses
barcodes
these are the places that i would look
here are the rocks that i'd turn over
i also hope to surface some of the risks
so that manufacturers can consider them
mitigate them and build better systems
and i'd like to help jurisdictions know
what to ask during purchasing
and to help them to secure the systems
that they already own
my overall goal is to help the defenders
not the attackers
i believe we can be mature and talk
about how to lower risk in critical
systems
without panicking and weaving things
into conspiracy theories
the process diagram is also a good
diagram of the attack surface
every point has attack potential
obviously
it's better to attack closer to the left
side before barcodes are printed
but there are attack opportunities to
edit or destroy barcodes later
even after the ballot has been tabulated
and stored
note that some systems perform ballot
creation inside
one device while others use separate
devices
position number two could represent a
usb cable between the tablet and the
printer
and some systems combine tabulation and
storage into one device
while others such as central count
tabulation split it up
and some all-in-one hybrid systems
contain everything from one to seven
inside a single device there are many
ways that we might launch attacks on
these areas
i won't spend time explaining them
because you've likely heard them
elsewhere
they include supply chain attacks
hardware tampering malware
machine in the middle attacks and remote
access
one important gradient in many of the
attacks we'll look at is access to a
printer
positions one through three are before
printing and have access to that printer
at position four usually the ballot is
under the voter's control
but there may be an opportunity for poll
workers to handle the ballot or the
voter may insert the ballot into a
device to verify their selections
as one vendor routinely demonstrates at
position 5
most tabulators do not have a printer
though some may add one to support
post-election audits by printing ballot
identifiers
ideally that printing would be in a
different color and restricted to the
margins
most tabulators scan a ballot and drop
it into storage immediately
but several all-in-one hybrid machines
have a single paper path
so the cast ballot passes a software
controlled printer
at position 6 on the way and finally
a printer isn't the only way to mark on
these ballots several
print on thermal paper applying heat
friction or scraping
will mark on those ballots you can make
a mark by swiping a fingernail across
them
and just like with any type of paper
ballot there may be opportunities for
manipulation during transport
storage and post-election audits
the first attack that we'll look at is
the most straightforward
printing a different barcode this is an
attack during creation
that applies to every type of barcode
it's the strongest attack
because the attacker has the most
flexibility about what gets printed
and has the least chance of detection a
hacked bmd
would be the ideal target because the
bmd has everything it needs to generate
barcodes
that's its regular job whereas the
printer
or a machine in the middle attack would
need to know the barcode format
the data structure and possibly
additional data to create ballots that
appear authentic
and pass any integrity checking measures
if the attacker replaced
only the barcode the voter would have no
way of detecting it
it would cast the ballot or the attacker
could create a much better forgery
by modifying both the barcode and the
text
since studies show that voters rarely
check the text
either way it's the barcode that gets
counted on election night
next we have a replay attack during
creation of a ballot
an attacker takes a barcode that they
don't like and they replace it with one
that they do like
this is different than printing any
barcode because the attacker
doesn't have to navigate all the
complexities of building a barcode
it's just mimicry of a known valid
barcode
one doesn't need to know the structure
or any secrets
it just has to know that one barcode is
favorable
and then recognize unfavorable ones to
swap out
but attack number two is suitable for
printer or machine in the middle attacks
at positions two and three where you may
not possess all the information that the
bmd has
next we'll look at two similar attacks
together
first if there's an undervote in a
contest where the voter has not voted
or has not voted as many votes as
they're allowed the attacker can add
their preferred vote to that contest
in these examples you'll see that the
voter could have voted for one
more candidate in each of these contests
as barcodes
those could be opportunities to steal an
undervote
it would net one vote for a favored
candidate or many votes
for a straight party contest
an attacker could also add a surplus
vote to create an overvote
if too many candidates are picked all
the votes in the contest may
become invalid in the example you can
see that each contest has one vote too
many
it would not net any new votes for a
favored candidate but it could subtract
votes for unfavored candidates
these examples use boxes not barcodes so
how would barcodes be affected
the inca vote that we saw with the
freedom vote tablet it's very vulnerable
adding a vote is as simple as adding a
square to the grid
on thermal paper i might be able to do
it with a small scraper
and because of their similarity mark
sense ballots have this same
vulnerability
one difference though is that marks
sense ballots have context
around the marks so the voter may notice
the change with barcodes they don't
code 128 may be vulnerable depending on
the implementation
it helps that each bar code is an
individual vote
the expressvote style of ballot may
leave blank spaces if rows are not
evenly divisible by 3.
esns has reportedly mitigated this in
future software
by filling in a row of x's esns also
records a count of the barcodes on the
ballot
i don't think that's a strong integrity
measure but it does offer a basic
defense
adding votes to the qr code would be
more difficult the format has a complex
structure
and each vote is not as distinct the
imagecast x implementation
also includes an hmac so changing
selections would change the expected
hmac and honestly
if an attacker has enough access to
create a matching hmac
then it's no longer distinct from attack
number one where an attacker can print
any barcode at all
instead of adding a vote an attacker
could modify a printed
barcode to have different values this is
not possible
with inca vote or qr codes one reason is
that modification must be
additive you can add a black mark to one
box
but you can't take away an existing
black mark
and we've already seen how qr code has a
complex format
and may have integrity checking on those
selections
however code 128 is potentially
vulnerable
the express vote implementation confirms
the count of the barcodes
but not their content let's look at a
code 128 barcode
it has a start character the data a
check character and a stop character
the six digit candidate identifier is
stored in pairs as three barcode
characters
in 093411 this is the 34 character
don't think of barcodes as being fat and
thin lines or spaces
think of them as 11 positions that can
either be black or white
two of the same color together looks
like a thicker line
34 is a good candidate for editing
because it has a lot of white space
where we can add black lines in 093011
the 30 character is not a good candidate
for editing because it has lots of black
lines already
and modification is additive we can't
remove those lines
but we can change 34 into 30 just by
adding two lines
but this new barcode won't be valid
after the data portion of the barcode
is a check character and its purpose is
to ensure that the barcode scanner
read the data correctly the scanner
passes the data into a fast algorithm
and gets a value between
0 and 102 which it encodes as the check
character
if the data doesn't match the checks
character then the scan fails
the check character for 093411 is
9. the check character for 09 3011
is 1. so if we modify the 34 to become
30 the expected check character would be
1
and it's not but the check character may
also be modified
if we add two lines to the nine it
becomes a one
and the resulting barcode becomes valid
if every row of candidate names is plus
two on the grid
the odds are good that these two
coordinate positions will be candidates
in the same contest
i wrote a program that runs through all
the editing possibilities around 31
percent of grid positions are editable
like this
of course not all columns in the grid
are used frequently
column one is often filled with ballot
instructions
column 9 is frequently used for the
first set of contests
and on page 1 side 1 48 out of the 99
rows
are editable to different positions some
positions have several possible edits
that could be made
093411 can be changed to 9 other grid
positions
1.0811 can be changed to 26
others have one or none
modifying barcode 128 barcodes has
constraints
but it is possible i found one example
at the bottom of the first column
where a straight party vote for one
party could be changed to a completely
different party
one modified barcode could change the
votes in many contests
a less refined attack would be to
invalidate barcodes
they could be printed as invalid but
appear perfectly real
they could be modified so they become
invalid or they could simply be defaced
the result might be that the barcode is
ignored
the ballot might be rejected or it might
create an undervote opportunity
imagine if you could invalidate one
whole qr code while adding an alternate
qr code
or it could even be used to hide
evidence of manipulation
we'll now shift our focus to attacks on
barcode scanners
using barcodes an attacker could
introduce a barcode
which modifies the scanner settings
commercial off-the-shelf scanners often
allow configuration by scanning special
barcodes
these barcodes are not secret they're
usually printed at the back of the
manual
these might be handheld scanners or
fixed mount scanners
thankfully most if not all of the voting
machine tabulators on the market today
make a full image of the ballot and then
use software to read the barcodes from
the image
software can be configured in other ways
so it has no need of these configuration
barcodes
so while a scanner reconfiguration
attack on ballot tabulation
is possible better targets would be on
other parts of the system
like the built-in scanners used for
ballot activation
or the ones used to transfer ballot
selections from a phone
what could be reconfigured there may be
as many as 50 different settings
an attacker could create denial of
service allow additional barcode types
to be scanned
change how duplicate or invalid barcodes
are handled
allow data to span several barcodes or
enable user-defined triggers so that
scanning a barcode
triggers an action like a macro
reconfiguration might help with our next
attack injection attacks
many of you are familiar with other
types of injection attacks an attacker
sends malicious input
that gets interpreted by the system a
barcode is a keyboard
as hari herste has frequently said
barcodes can include characters commonly
used for injections
they can shift to different character
sets to access special characters
unlike reconfiguration attacks injection
applies equally to any computer process
that doesn't fully sanitize or allow
list the data input
both hardware scanners and software
scanners may be vulnerable
one might be able to use code 128 with
less than 10 characters
to send the keystrokes that are needed
to exit the running program and launch a
command line
interface qr code and other formats
may hold hundreds of characters enough
for a small program
this is an attack that could have a lot
of impact
the last set of attacks we'll look at
are disinformation attacks
disinformation comes in all shapes and
sizes but barcodes are vulnerable
because of their lack of transparency
they're not verifiable by the voter
even if the voter has a barcode reader
in hand
certainly some voters ignore them or
just accept them
but many voters routinely express fear
uncertainty and doubt about them
the seed for disinformation to latch
onto is
if your vote was in a barcode and the
bar codes are what the system counts
how do you know your ballot was cast and
counted as you intended
now my opinion which is only backed by
subjective and anecdotal evidence
is that bar codes provoke mistrust more
than paperless dre systems do
that's odd since paperless systems
should be less trustworthy
but paperless dre systems don't wave any
red flags during the voting process
a voter is not provoked to consider how
the voting machine does its work
barcodes raise the question of trust
with a prominently placed visual
voters can't help but notice unreadable
glyphs on their ballot
and they have to decide how they feel
about them it sets the stage for
disinformation attacks all right
let's look now at how defenders can
detect and mitigate these types of
attacks
we'll start with some things
manufacturers of voting systems could do
manufacturers can improve the overall
security of bmds protect
barcode scanners like they would any
other port and sanitize and allow list
all data from barcodes to protect
against injection attacks
and they can prevent scanner
reconfiguration in the field
their systems should not re-expose
printed ballots to printers
there should not be a single paper path
that gets reused
most importantly they should design
systems to confirm the integrity and
authenticity of data coming from
barcodes
and in a strong way that considers
adversarial attacks
that may mean public key cryptography
and it may require working out good key
management practices
and working to make barcodes more
transparent and interoperable
would help to make them more trustworthy
jurisdictions can do a lot to protect
the systems they already have
they should think first about resilience
planning a huge vulnerability
is that a malfunctioning ballot marking
device can't write barcodes
and a malfunctioning tabulator can't
read them if a machine needs to be taken
out of service
jurisdictions should have a backup plan
that's ready to go
jurisdiction should ensure that ballot
marking devices have good physical
security and chain of custody measures
especially during voting machine
sleepovers and a polling place
don't assume that the bmd is less
critical than the tabulator
they should encourage voter verification
of all bmd printed ballots
to detect errors and anomalies this can
be done with signage and with poll
worker prompting
they can't and won't detect everything
but studies show that it does help
poll workers can be trained on potential
barcode issues and jurisdictions can
advocate for legislation and policies
that define the official vote this is
important because one issue barcodes
create
which hasn't existed previously is that
a single piece of paper contains
two separate representations of a vote
the barcode version
and the text version if they match
there's not a problem but if the
barcodes in the human readable text
don't match
it needs to be legally clear which one
should govern in most places the law
doesn't consider that possibility yet
and accordingly any recounts of the
ballots
should use the text as the official vote
and by far the best way to mitigate any
attacks that change election outcomes
is with post-election audits they
prevent tabulation errors of all kinds
the gold standard is risk limiting
audits which are efficient
but also tied to the margin of victory
so that you test more ballots when a
race is close than when it's not
any audit of ballots with barcodes
should examine the human readable text
on the paper
by the hand-eye method and not the
barcodes the more contests that are
audited
the more protection an election has
so in conclusion are barcodes on ballots
bad
as we've seen barcodes introduce new
attack surface and new risks
many of those risks can be mitigated or
managed but we have to actually do it
and there are gaps at the moment
the hardest issue to mitigate is the
lack of transparency
this is the primary reason i'm
personally not in favor of barcodes
it's not the risks or the attacks
barcodes make voters uneasy
i've heard it from many voters they
worry that the system could be doing
slight of hand or pulling a fast one on
them
you may have that gut feeling yourself i
understand it
to me that runs counter to our goal of
building trust in elections
there are two voting system
manufacturers who do not put votes in
barcodes
heart and clear ballot so barcodes
aren't a necessity
in 2019 the state of colorado announced
that after 2021
they would no longer certify voting
systems that tabulate using barcodes
as a result dominion modified the image
cast x bmd used in colorado
so that it fills in the ovals on a mark
sense ballot
instead of using a qr code like the
ballot that we deconstructed
i realized security is not the only
consideration in making a choice like
colorado's
but with one action it took barcodes off
of the ballots
and reduced all of the risks that we've
looked at today
thank you very much