welcome to my talk on smart meters i'm
hacking infrastructure
and so should you in this 25 minute talk
i'm going to walk you through all the
different things i've been doing the
last couple years
to reverse engineer smart meters
particularly the ones made by landis and
gear that are used here in texas
by the power provider encore i picked
them strictly because they're the one
that was on the side of my house and i
was interested in learning more about
them so nothing in particular about
those two
or the the focus of anything other than
it's the meter i have and the power
company i use
you'll see uh all kinds of circuit board
reverse engineering
protocol analysis uh some software
reverse engineering using binary ninja
that i've kind of just
begun recently there's a number of
youtube videos i also have on this that
kind of
was the precursor to this talk and a
wiki
um called richesom and so there's a lot
more in-depth data and
in-depth protocol stuff there's a
discord channel that i have so
a bunch of ways if you're interested in
getting involved in this that you can
as a little background i've been into
reverse engineering a long time
so i've watched a lot of talks online by
different people chris tarnowski reverse
engineering silicon
um it's real fascinating to me chris
gerlinski and the work he did reverse
engineering uh cable tv
uh pay tv box that was fascinating um
and travis good speed obviously a ton of
work out there
poc or gtfo um other things like that
that he's done and
and a talk that he gave that was really
fascinating to me that was
called in praise of junk hacking and and
the premise of that talk if you don't
watch it i suggest you do
is that you know these critical
infrastructure devices
they all contain off-the-shelf
components and so
you know things like the processor
that's used the
transceiver chip that's a cc 1020 made
by texas instruments that's in here
those are all used in other devices like
the girl tech imme
that him and michael osmond worked on
and things like that and so his premise
was if you reverse engineer
things that kind of are in products that
don't matter but they're used in
products that do matter
you're essentially pointing out and
finding flaws in critical infrastructure
or things like
that without um without the undue
attention that comes from reverse
engineering critical infrastructure
and and that talk is uh quite a few
years old at this point
and at the time i think that that's you
know is a really sound way to go about
it
um and i think also if you just don't
want to attract any attention that's
fine as well but the the challenge is
these devices run you know kind of our
our whole economy and
and with the way ransomware is and and
things like that now
you know you used to have to steal
something of value and then sell
something of value and and that's how
you made money you know you robbed a
bank
you got the money uh you hack into a
place you get the credit cards
what we see now with ransomware and
pipeline hacking and
and even hacking corporations is that
you don't have to steal anything of
value anymore
you can just stop someone from operating
and
they will pay you a bunch of money uh to
restore their operations
and and so you know that's a big
motivator for me now
looking into these devices and
publishing things openly
so that we're not under ndas there's
nothing stopping us from having a
conversation about these things
all of these have been purchased legally
off of ebay there's a whole market for
this stuff out there
and it's fairly cheap to pick it up and
you'll see um
i've bought a lot of this stuff you know
as we pan around my office here
this is kind of just a piece of it as
well like there's even more devices that
i have because i've been doing it for a
number of years and
it just doesn't cost that much to get
this hardware and it's super fascinating
to try to learn every single piece about
it you're going to see that
there's a whole geographic routing
protocol that's used with this it's a
massive mesh network
where all the data hops between meters
and up to routers that
i happen to look like this
and so we're going to look at all this
stuff
i've been publishing it all on on
youtube and and on the wiki as well
and doing a bunch of protocol analysis
and so that's what we're going to look
at today for the next uh
however many minutes are left at this
point and so i hope you enjoy it
thanks a lot i encourage you if you're
interested in this step as well join the
discord channel
check out the wiki create an account
contribute stuff that you learn
i think it's a fascinating topic to look
at it's a fascinating device and these
massive networks of you know millions of
devices on
some kind of a 900 megahertz mesh
hopping network that you probably
haven't heard anything about and with
the advent of
software-defined radios like this hack
rf with a portapak or usrp
b200s um even rtl sdrs
you can analyze this frequency hopping
data and and learn more about it and so
that's what you're going to see that's
what i've been doing i hope you enjoy
the rest of this
okay so let's set the stage with some
gps coordinates plotted on a map
these are all the smart meters that are
between where i live in downtown dallas
as you drive up and down the freeway
with an antenna on the car what it can
receive the height above the ground is
how long they've been running since they
last lost power
so you can see in this there's the meter
id and it's been running for 1767 days
now what made me think that the meters
would even have gps coordinates it comes
from this paper here
that i read i found on the ieee and it
was written by some of the people that
work at landis and gear and it really
outlines the entire system and
how it functions so it gave a really
good framework
as i began digging into these meters to
try to understand
how do they even work and this right
here is the big part
it says there's a geographic routing
protocol
and that really struck you know that
peaked my interest like a geographic
routing protocol
what does that even mean and so as you
read it there's a piece here i'll look
over to the side
and read what it says says the rf mesh
system supports peer-to-peer
communication by employing a routing
scheme which utilizes the geographical
coordinates latitude and longitude
of the communicating nodes by nodes they
mean smart meters
as you can see here it shows all the
meters they send their messages they
relay in between them so it really forms
a large mesh network
but then it has to get off of that mesh
and it goes up to what they call a
router
like the device i showed earlier that's
actually a collector we'll get into that
in a little bit
then it goes to what they consider kind
of a traditional collector this is a
device sitting inside of a substation
usually with a big pole and two to four
antennas on it
now i just want to give you a little bit
of a framework of
how the packets are constructed these
are called five five packets
they're basically broadcast once a
minute and they contain
some kind of universal information they
have what you call a wan id
which in this case turns out to be the
gps coordinates that i'll show in a
moment here
and then the lan id which is the id on
the front of the meter
and some other information that comes
across that we've been working on
decoding now hunting online a bit
searching every single document i can
find which there's not a lot it's mostly
on the fcc's website that you find this
stuff
there was some really good manuals that
gave some information about how to
configure the devices and what they
showed was people
inputting gps coordinates and something
called a color code in the north south
and this kind of information and then
the output
would be this other string of hex values
that looks a lot like the wan id the
more i started searching around
so then the hunt was on to figure out
how do i convert this input data
the gps coordinates to that output data
so that i can go back
the other direction so i found eight
unique values
and i organized them put them in a nice
little list you see here and organize
them by the latitudes from smallest to
largest
and the idea would be to try to look at
the data on the right-hand side the
output
and understand you know if i'm looking
at that data
does it correlate at all with the
smallest to largest you know what what
can i find about it
and it stumped me for quite a while but
like all things i found with these if
you just keep breaking things down from
hex to the individual bits
and then look at the bits things start
to become much clearer
and so what we can see here is that i
didn't know much but from reading i knew
that this idea of a color code that i
put a c in front of here that color code
was 31
32 unique values so five bits basically
and on the right hand side of this you
see that everything's zero
except for one bit that's set there
meaning two
and so i knew probably the last five
bits are going to be the color code so i
can kind of strip them off
so now as a matter of just looking at
the rest of the data and then looking at
the pictures
that were from the manual to see okay if
i take a bit
for the north-south indicator a bit for
the east-west
what does that leave over for the
latitude and longitude
and then try to convert those values to
see okay do they come back to
the latitude and longitude values that i
see there and how do i do that
conversion
so it came down to a little bit of math
that i boiled down to a formula right
here
there's a much longer video i have on
youtube about decoding the gps
coordinates
so if you're interested in exactly how i
figured all this stuff out
uh feel free to watch it but otherwise
know that the gps coordinates are
embedded in the meters they're
transmitted once a minute
and this is how you decode it and all
this data is available on github
in a block that i wrote for gnu radio
so this is the full gnu radio flow graph
that you see here
and i can't show you this without
mentioning jacob gilbert who did a great
presentation a little while back
at the gnu radio conference and he shows
this whole framework this frequency
hopping
spread spectrum toolkit that they wrote
that basically grabs all the individual
transmissions
uh packetizes them in what they call the
pdu
and then sends it on and so it's using
this framework that i was able to grab
all these frequency hopping packets send
them into this block that i wrote
and then output the data that ultimately
gets sent over to a python script
that decodes and dumps the gps
coordinates
so what you see here is gnu radio
running it's grabbing packets receiving
them
i dump it on this terminal on the left
that's the debug output showing the raw
packets and then i start up the python
script
and i basically have it point to a tcp
server that's running
inside of gnu radio in one of the blocks
that's there
and it grabs that data that's coming
across any of the five five packets that
contain that gps information
and then it dumps it and then ultimately
i take it to a csv file and send it into
google
earth which is how it was visualized
so let's switch gears and take a look at
this collector i had owned the thing for
a while
but i literally hadn't even opened it up
i got it i set it on a shelf because i
was so busy playing with meters
and someone tweeted that inside there
might be an ssd drive and you know
and so let's take a look inside and see
what's there i have a bit of a longer
video that really dives into this and as
a reminder
this is one of the devices it's like one
that would be mounted up on a pole
that's getting all the the data back and
so inside you see there's a really large
battery
that you know the thing uses when the
power goes out so that it can keep
relaying
messages back from the power meters they
can also send messages
for you know 10 15 seconds after power
goes out
and so as i pulled everything apart you
see there's one of the radio modem
boards
and then underneath this panel right
here
yeah it's an ssd
so at this point i'm thinking jackpot
and so i pull everything else out you
know i want to do a full teardown
there's pictures on the wiki
of all this stuff that you can really
see detailed shots of the
small single board computer they use
it's intel based
and all the other components and this is
really the back plane board that i'm
taking out
it has all the power supply components
and also a cellular modem
that's inside of this device that they
use to relay the information back to the
power company
so i took an image of that ssd and i
mounted it
in linux and what you see here is the
the thing runs windows if you can
freaking believe that
um windows embedded seven the whole file
system's there
um there's a folder called collector
conveniently
that has all the files and everything in
it and it's all written in dot-net
and it doesn't even appear to be
obfuscated in any way i also was able to
take the same image
load it into a virtual machine and boot
it up and run it
and so here you can see that there's the
cellular modem that it's looking for
it's not connected
and you can also see the terminal window
showing the version of windows that
that it's running now after i let it sit
and run for about
10 minutes i'd walked away to you know
get some coffee or something
i come back and this program is running
it only seems to run if there's no
serial ports attached the minute it can
actually talk to a serial port
it just shuts down and reboots
constantly trying to find
probably the modem or the the cellular
modem or the radio
that's inside this device and it can't
find either of them so let's pause on
the collector here for a little bit and
switch back over to the meters
themselves and take a look at some of
the reverse engineering
that i've been doing there early on when
i started looking at them
i realized i needed to create some
schematics of these so i'd have some
kind of reference to understand where
everything's connected how it's
connected
and where i can supply a power so i
don't have to run them all off at 240
volts
so here you can see some of the
schematics that i've created of the
different processors the m16c and the
terridan
if we back out what i did a bit is
i bought a ton of these boards on ebay i
found someone that was selling a bunch
of them that
just weren't in meters and i stripped
off all the components on one
so i'd have something to have as a
reference i also took advantage of
having
all these different meters and boards
i've purchased to dump out all of the
rom chips
so that i can compare all the different
files and see
you know is there any settings what's
different between them
what can i learn about how these things
function by comparing a bunch of
different data files
now i also started looking at these
integrated wingate radios
and there's a bunch of different
versions of them but this old one is all
through-hole components
and so it's much easier to analyze and
just literally read the firmware out
and there was these jumpers on it that
you could set they would put into a test
mode
and in that test mode i could see a menu
that would show the status of the
jumpers
and so on the meter itself i hadn't
found a way to enter
this test mode but someone else had
dumped a memory chip or tried
and erased it on accident and what they
found was that when it booted up
it would actually dump you into this
same kind of test mode menu
so i did the same thing and inside there
i saw that it would show the status
of the jumpers and so going around this
chip with the resistor
and trying to set some pins low i found
the pin 89 on this m16c
would actually put it into test mode
without having to erase the
the flash memory now there's a bunch of
other cool menu entries in here to let
you do things like read out different
parts of the memory and
and test a bunch of things but there's
one that i particularly like
that lets you receive rf data from
a network around it and so you have to
specify the crc seed
which i was able to figure out for the
encore system around me
and some other information like pick a
channel but when you do that
then it receives some data and i don't
care so much about the receiving of the
data
because i already built a new radio
thing that that does that
but what's more interesting is how they
organize the data on the screen
and even what is in the test menus like
what are the things that they care about
and so that's much more interesting to
me to see the different types of things
they show
and what they're concerned about than
the actual data itself
which i can just read directly out of
these chips now also a while back
i managed to dump the firmware out of
one of the meters that i have
and so i have this m16c firmware i
loaded into binary ninja
and it was a large file and reverse
engineering assembly isn't my uh
full-time job you could say but i also
managed to get the bootloader out
and the bootloader is only 4k and it's
much easier to understand
and so i've been reverse engineering
that a bit to understand how they
load the key from memory that they use
to determine whether you access the chip
or not
and to potentially look for flaws inside
of the bootloader itself
so i've shown you a bit of what i've
been working on up until now
let's talk a little bit about what i'm
working on in the future and what would
also be interesting to figure out
so you've seen that they use the
frequency hopping pattern
in the 900 megahertz band what would be
real interesting is to figure out
well how are they hopping around and how
do they choose what frequency to hop to
next
every 700 milliseconds they hop to
another frequency
and i believe it's partially based on
the id of the meter
and some other information and they
discover each other even though they're
all hopping around
on these different frequencies so how
they hop around and what frequency they
choose at what time
would help us lower the cost of the
hardware that's needed
to analyze and receive all the messages
on the network
so far everything you've seen is me
receiving data
from the meters and trying to figure it
out but would be real interesting
is using the faraday cage transmitting
back to the meters
trying to get them to accept packets as
valid
and see how they respond so not
necessarily just
random fuzzing but actually crafting
valid packets with checksums that are
correct
and also the timing data and this other
information
and and really sending it a packet that
it believes is coming from the network
and see how it responds and then change
things about that packet while it's
still
signed properly and see how does the
meter react to that
also some of the messages that are seen
on the network
start with a d5 i only showed you the
five five packets
the d5 appear to be a routing protocol
for sending messages across
where the meter identifies itself and
then identifies the address that it
wants the message to go to
across the network what if we could
craft our own packets
and send them on to this network and
travel across it to another device
without ever actually going to the
router into the back end
you know could someone create a message
and a network of devices that ride
on this mesh network without actually
causing havoc or
or doing anything so benignly traversing
this network
without actually leaving the local area
up to a router and back to the head end
that's interesting as a thought
experiment also just the process of
discovery
so while looking at the different iwr
radios the integrated wayne gate radios
i noticed that i actually had five
different or four different generations
of these radios i had
second third fourth and fifth generation
radios
and by analyzing the data it's sending
what i noticed is the second third and
fourth generation
all use the same sync word that it sends
initially so that other meters receive
that message
and process it but in the fifth
generation they change the sync word
probably so they can add additional
features or to change the
the protocol and the rest of the message
still seem to be craft the same
on the meters around here but the local
meters would ignore it
since the sync vi the sync violates the
start and stop bits that are used
traditionally by the previous
generations
so that tells us that in the fifth
generation fundamentally
something changed they added some new
additional functionality
that'll be worth digging into also i
didn't talk about this thing much
but inside every single meter is the
ability to shut off power that's right
they can send a wireless message to any
single meter and shut off power to that
location
could we do the same thing you can also
control it through the infrared port
i've seen it in the menu items
that show up when i'm logged into the
menu
on the meter itself i haven't tried
playing with that yet
but i will in the future but also on the
rf side
what kind of packet comes across that
turns power on and off
i've been capturing a lot of data
perhaps it's already in there and i've
just missed it and it's waiting to be
discovered
and finally like i said all this
information the new radio block
it's on github you can see up here it's
also documented on the wiki
the wiki is always a couple steps behind
what i'm discovering just because
documenting everything and discovering
it and creating videos
all takes a fair bit of time but i do
update the wiki
and anyone is can have an account i
encourage you to create an account and
anything you find or any other meters
you work on you can publish the
information there as well
and finally links to all these things
are off of their chessum landing page
you see it right here at the bottom you
can click to go to youtube
discord smart meter data
and any other links that are of
importance i'll include here it's an
easy place to
to find
[Music]
so
[Music]
you