hi there thanks so much for having us uh
to co-present my name is jen goldsek i'm
the ceo of the digital medicine society
or dime for folks who may not know dime
we are a 501c3 non-profit organization
dedicated to advancing the safe
effective ethical and equitable use of
digital technologies to improve lives i
am delighted to be co-presenting um with
my colleague my longtime collaborator
and great friend jessalyn dunn jesse do
you want to introduce yourself to
everyone sure thanks so much jen um my
name is jessie lynn dunn i'm an
assistant professor at duke in the
departments of biomedical engineering
and biostatistics and bioinformatics i
run the big ideas lab at duke um and i'm
thrilled to be presenting at this venue
and presenting alongside jen
fantastic so the way we'll run this
today um i will start by um presenting a
little bit on sort of current state of
the field and specifically digital
clinical measures um then what i'll do
is i'll hand over to jessalyn to take us
through some of the science that we use
to ensure that when we are using digital
technologies digital tools to measure
health
we can be certain that those measures
are trustworthy then jessalyn will hand
that to me i'll keep it short at that
point
and we'll just reflect on what needs to
happen to take all of this to scale and
importantly jessalyn and i actually
recently collaborated um on some uh
research we have some data on the state
of the field and in particular uh
security research um that we thought it
might be interesting to share so that's
how we'll close out so with that in mind
let me go ahead and start sharing my
screen
jessalyn can you see that
yes
fantastic all right so let me go ahead
and dive in so we're going to be talking
about digital clinical measures today
and specifically what's the current
state of the field and how can we
anticipate them evolving in sort of what
hopefully will soon be a post-covert
error
what's interesting is if we look to the
um if we look to the press for example
we'll see that many of our sort of uh
terrific scientific sort of journalist
colleagues will say that healthcare is
undergoing a monumental shift towards
remote patient monitoring during the
last year three of the five biggest
companies in the world have announced
new remote monitoring products we've
seen amazon put their halo out that was
launched last uh last summer um google
now owns uh the fitbit sense they are
actually ha a cleared technology um so
in some ways they are a regulated device
their ecg app is a clear product
although that is the only part of their
sort of measurement suite that is
cleared
there's also apple who have um developed
a phenomenal amount
of health measuring capabilities through
their smart watch um they also have a
clearance specifically for pulse ox
monitoring
let's think about how these tools have
been used recently though and i think
this is really important for us to pause
and think about um
there's been a lot of use of technology
in healthcare but the digital clinical
measures piece in particular um where is
that um in its development
during the pandemic at its height we saw
telehealth increased by nearly um 1200
at the same time of those telehealth
visits only 11 of those relied on remote
patient monitoring
now there's a couple of things there
first of all i actually don't think by
any stretch of the imagination that
every single telehealth encounter needs
to be based on or reference um sort of
digital clinical measures data we don't
have to be reliant on remote patient
monitoring for every telehealth visit
however 11 does feel low
kudos to our clinical colleagues all of
this care during the pandemic delivered
by telehealth they turned on a dime to
be able to deliver that care so there's
definitely some aspects there that
perhaps those tools and technologies
weren't deployed and weren't available
but that 11 number does seem low it
shows that there's a lot of work to do
to integrate these tools
safely effectively
into clinical workflows
i think it's also worthwhile um to look
at some additional data we collected and
published um around how those uh digital
data are being used when remote patient
monitoring um
uh is happening
really interestingly um when those 11 uh
when we're uh sort of leveraging that
data we found that the vast majority of
times
it wasn't that um clinical care was
actually taking advantage of the
continuous nature of those data streams
rather they were being referenced and
they were being used in exactly the same
way that we would treat a stock data
point
captured in the clinic
what this tells us is on the healthcare
side there is enormous potential but we
haven't yet harnessed the power of these
technologies through a redesign of the
clinical workflow flow through truly
monitoring patients outside of the
clinic during activities of daily living
we aren't tracking the monitoring to see
if there's a spike or a within patient
change that should trigger some outreach
that should trigger additional care
we're not there yet it's certainly um a
powerful way that these tools will
transform the way we think of healthcare
but it's important to realize where we
are today
so that's over on the um healthcare side
of the shop where are we in the sort of
clinical research side where are we
specifically in terms of medical product
development
um if we go back just a year and a half
in october 2019 my organization dime
launched a crowdsourced library of
digital endpoints being used in
industry-sponsored trials of new medical
products and we were absolutely
delighted to see that there were 12
sponsors
um who were sort of actively collecting
these measures and that 34 unique
endpoints were
being used to answer clinical questions
about uh new therapies
we were so excited in fact we went ahead
and we published an op-ed um in stat
news saying you know this is not the
stuff of the future this is here today
but when we actually look just 18 months
ahead the growth has been absolutely
astronomical what we can see is we've
gone from 12 to 62 sponsors relying on
these digital clinical measures to make
decisions about whether we take a drug
to market
we can see that over 50 so over half of
these um
measures are being used uh in critical
phases of drug development and we can
also see that the growth in the number
of measures themselves has gone from in
uh
from 34 up to 207
absolutely extraordinary growth not only
in the number of measures but also in um
sort of
the trust essentially by many many more
organizations to
rely on the data from these digital
clinical measures in their decision
making in their product development
pipeline in their r d
um if this data is interesting to you
this is all open source and you can
access these data on the dime website
these slides will be available and we'll
make sure you get all of the links
i do think it's worthwhile pausing for
one second though what's interesting as
you mine the database is um you'll see
that every single one of those 207
measures is unique and this while the
growth is impressive this does give me
pause
we've got to think about measuring
things that matter i don't think that
there are 207 unique measures of health
that are critically important
we see over 50 unique measures of sleep
certainly sleep is a complex um a
complex sort of uh domain of health it's
unlikely that there's just one measure
that will account for it perfectly in
all therapeutic areas but the idea that
there are 54 measures that warrant
development i have questions about the
same is true for physical activity i
think we've got over 70 measures of
physical activity in this library and i
think it's really important that we're
focused not only on our capability of
measuring health but actually making
sure that we are measuring things that
matter and we'll return to that in just
a little bit
and so you know jeslin and i talk about
this a little bit but you know
the reason that we're excited about sort
of the field of digital clinical
measurement generally is
we imagine because these measures
because our understanding of health can
be developed outside of the walls of the
clinic because it can be developed
during a patient's everyday life
we can imagine a future state of
healthcare where
healthcare is good because we keep
people healthy and we keep them out of
the doctors healthcare is no longer
measured by how well are we able to dive
in and add the assist once someone is
sick once they need to come to the
clinic or come to the hospital
and i wanted to give a couple of
examples just to underscore this so
first of all in current state if a
patient needs a bone marrow transplant
in the two weeks preceding that visit
they have to come into the clinic
roughly five times simply to have vitals
measured to make sure that they are
stable and healthy enough for surgery
there is absolutely nothing right now
technically stopping us from measuring
all of those vital signs with a high
degree of certainty while that patient
is at home without dragging them into a
clinic while they're immunosuppressed
without asking their care partner to
take more time off work taken and bring
them into the clinic without sort of
adding to the stress and the time and
the fatigue
that these continual clinic visits rely
on and to me that's actually something
that at some point it becomes unethical
to to to choose not to use those tools
to keep bringing that patient in for
fee-for-service lab visits as opposed to
simply monitoring their status at home
as they prepare for that surgery
um another example is in duchenne's
muscular dystrophy and for folks who
might not be familiar with the natural
history of duchenne's um it's
a pretty awful condition that affects
primarily boys and young men um the
natural history is diagnosis is
typically around seven years of age
unfortunately these boys are usually in
a wheelchair um
struggling to sort of walk independently
by the time that they're 13 and
unfortunately life expectancy is in the
early 20s
so
how are we currently measuring this we
use um the six minute walk test um that
systematically excludes 60 of the um
population of those boys and young men
from participating in any kind of
research that means that we aren't
developing medical products that target
those individuals and whether in
research or in care we aren't thinking
about
the pieces of their condition that maybe
matter a lot to these boys we're not
thinking about whether they can text
their friends play video games whether
they can feed themselves whether they
can do their homework whether they can
take themselves to the toilet their
dignity
we have the capabilities to measure that
using things like smart watches smart
rings and yet we still rely on the six
minute walk test and the last example
that i wanted to share to really
underscore why what jessalyn and i are
talking about today these digital
clinical measures of health are so
important
um parkinson's disease
unfortunately the number of people
affected around the world by parkinson's
is growing rapidly in the u.s most
people with parkinson's are diagnosed
incorrectly four or five times before
they get that correct diagnosis and then
once they are diagnosed due to the
maldistribution of specialist
neurologists and the patients who need
them 40 of parkinson's patients pass
away before they ever seen a neurologist
imagine the power of
a suite of uh sort of digital clinical
measures measuring things like tremor
freezing of gait measuring things like
um
uh cognitive decline measuring uh tenor
and toner voice
all of those data could be collected in
the home of the patient and then we
could use telehealth to connect those 40
of patients who never see a neurologist
to a specialist who can more
appropriately care for them it would
also allow us to develop a disease
modifying treatment which to date we've
had no luck doing that's why i think
this is so important that's why we have
to focus on those measures that matter
we can harness all of this data right
now those sort of continuously captured
currently invisible data points um using
uh using these tools
but what we have to do is make sure that
we are generating powerful meaningful
signal and not simply adding to the
noise
and to make sure we're doing that i'm
going to go ahead and stop sharing um
and jessica lynn who's lab absolute
experts in doing all of this work um
she will dive into some of the tools
techniques and science that they use to
make sure that these measures are
trustworthy
thanks so much jen um this was a a
perfect um tee up to what what we're
diving into next um
so i'm just gonna turn on um full screen
and hopefully you're seeing
no notes and just slides
perfect
great
all right um so
jen said it perfectly there is immense
potential for
these digital technologies that is
currently going untapped for the most
part um so in my lab we're called the
big ideas lab it actually stands for
something
which describes what it is that we do
which is biomedical informatics data
integrating data engineering and
analytics and with that we're trying to
take these existing technologies and
develop new infrastructure
to capitalize on that data for early
detection intervention and prevention of
disease
i don't think at this point in time we
really need a reminder about how
important this is we're living in a
global pandemic um and what's
fascinating is that there have been many
existing and ongoing efforts to try to
harness and leverage our existing
digital infrastructure to curb the
spread of cobit 19.
um there are a lot of different areas
that digital infrastructure can be
useful for so i just gave a few examples
here on screen
um where you know our current standard
of
uh care or standard of
exploring who might be infected is
contact tracing so getting on the phone
and calling up people and finding out
who they've been in contact with there
have been a lot of efforts to try to
automate this process because we all
carry smartphones those smartphones have
the ability to know when they're near
other smartphones or other um
technological systems and so this is one
area where we've we've realized the
power of the tools that we use passively
in our everyday lives
um there are many many other areas and
today
i will give an example of how we can use
wearables um and you know wearables can
be useful also in a variety of different
ways
um from the detection and monitoring of
infectious disease or chronic disease
all the way to the interventions to
deliver information at just the right
time to the right person
again there are so many opportunities to
apply this technology i'm going to focus
here in my talk on early detection but
you can imagine how jen mentioned the
possibilities for at-home monitoring for
a variety of different disease types we
can also expand this to public health
monitoring because if we have a huge
population already wearing devices or
carrying devices there's a ton of
information that's being passively
collected that could be leveraged to
realize where there are hot spots of
infection outbreaks before people are
exhibiting symptoms
um so again so many opportunities for
this technology so first i want to
explain a couple of terms um to dive
into some vocabulary when we talk about
digital biomarkers what we actually mean
um
first i like to back up and just revisit
this concept of a biomarker because many
folks who are working in this uh space
maybe don't have sort of the traditional
view of
what we think of a biomarker when we
think about a clinical use case so
biomarkers typically have diagnostic
capabilities and can also help us to
learn new things about physiology so you
might think about an existing clinical
biomarker like a blood glucose
measurement and that allows us to know
who may have diabetes or not how bad
that diabetes may be and if we give
somebody a treatment like metformin we
can track how well that treatment is
controlling the diabetes status
we also through
many new technologies allowing us to
measure large quantities of biomolecules
can discover new biomarkers that are
associated with disease and that allows
us to learn something new about
physiology so for example we might see
that there is a gene that is associated
with diabetes onset in young people that
is a biomarker that can then be used to
predict who might later come down with
diabetes
so we can extend this concept of a
biomarker to a digital biomarker which
means that rather than taking a physical
sample of blood or urine we're actually
taking a digital sample so we're
measuring something about physiology for
example heart rate from a vital sign
monitor that we can transform into an
indicator of health
so what that looks like on a very
pseudo-mathematical level is that we
have some uh outcomes and we develop
this function of these digital measures
and so these outcomes might be more of
our traditional
targets like blood glucose for example
or other clinical measurements it could
be survey data it could be illness
events
and the input data that we have here can
be anything that is collected digitally
from a smart watch a smartphone from
digital online traces
um you name it we have a ton of passive
sensors in our environments and that
information can tell us things about our
health
and so this isn't just a futuristic
vision there are a lot of digital
biomarkers being developed in practice
and this slide is a little bit old um so
it's just an example of what already
exists today
but we and others have been developing
digital biomarkers ranging from
infection detection
um all the way to chronic disease
detection and monitoring and you can see
that a lot of these digital biomarkers
have commonalities in the types of
sensors or measurements that they use
many surround heart rate and blood flow
many surround temperature measurements
or movement measurements and a lot of
the machine learning algorithms that we
apply to that data also have
commonalities um so we use
um
basic regression strategies we use
random forest neural networks and we
bring these common elements together to
develop these new predictors of health
and disease
so with that i want to pivot for a
moment and and introduce a little bit of
vocabulary that will help us to kind of
get through the rest of this talk um so
we worked extensively with um
jen and the team at dime and a team of
experts kind of from around the globe on
developing a common language that we
could use to describe these digital
clinical technologies that are enabling
us to capture these kinds of
measurements so
this idea of a biomet is essentially a
connected digital medicine tool and when
i say connected i mean that it has some
sort of wireless or even a wired
connection but it sends data from its
own location to somewhere else
and it processes data that is captured
by a mobile sensor and that there are
algorithms that are on board that
translate this these sensor measurements
into measures of behavioral or
physiologic function
so this sounds like a very very detailed
definition and it is because when we try
to think about these different
technologies that exist in our world
today it's hard to know what we classify
as a
um a wearable or a mobile device it's
hard to know what's regulated by the fda
versus what is categorized as a wellness
technology and so by introducing some of
this very specific language our hope is
that we can start to develop standards
surrounding these biomats which i'll
talk about in a second
um but one of the things that i like to
mention here is that a biomat is not
just what we see on the outside right
it's not just a fitbit or an apple watch
or a garmin device that's sitting on
someone's wrist but there's a whole lot
going on under the hood and i think for
this audience in particular it's really
important to recognize all of the data
transformations that can take place
on the device itself and then in the
process of transferring data from that
biomet to wherever that data is headed
um and so there are a lot of
considerations for
what is going on with that data are
there transformations that are taking
place are we compressing data are we
down sampling data
are there security and privacy concerns
at these onboard transformations or at
transformations or processing to push
data to somewhere else
so there's a lot happening under the
hood
and in order to make accurate digital
biomarkers that can outlast the product
life cycle it's very important for us to
understand how each of these stages of
data processing affect the final digital
biomarker
so then that begs the question of how do
we evaluate these biomets and digital
biomarkers so if we see that
a smartwatch company is advertising
their heart rate detection algorithm as
clinically validated what does that mean
so one of the major challenges here is
that there really is not a standard
right now for um
for the terminology surrounding the
evaluation of these these biomets and
these digital biomarkers and so with the
dime society this is what we set out to
develop
um we again develop some of these more
systematic terminologies like the data
supply chain which i was showing you on
that previous slide
shows us how data flows through the
biomet
and then
these ideas of uh evaluation processes
so we have verification and validation
that we've organized into a really nice
framework that we refer to as b3 that
helps us to determine whether a
technology or a biomet is fit for
purpose
and so when we say fit for purpose we
mean that
the final
experiment or
implementation in the clinic fits the
way that
device or that biomet or that digital
biomarker was evaluated
so so what this v3 framework entails is
essentially a verification process an
analytic validation process and a
clinical validation process
the example we like to give in this case
is of an accelerometer where we might
have an accelerometer that we would
place on a shake plate in the lab and we
expect that the rpm that that shake
plate is set to is what the
accelerometer itself would measure
now ultimately our goal might be to use
this accelerometer to distinguish
between health and disease of a specific
disease
but in order to do that we want to know
what is the movement that we're trying
to capture so let's take ms for example
and say that we want to use step count
to capture who's likely to have ms and
who's not
well we want to make sure then that this
accelerometer can actually accurately
capture a step so what we do is we have
a gold standard evaluation process like
a person actually watching a video and
marking steps and we compare that to the
accelerometer and that's the analytic
validation it means that when we put the
accelerometer on a human being it
measures what we expect it to measure
and then finally we have that level of
clinical validation which is if we put
this technology on a patient are we able
to distinguish between a disease state
and a healthy state for example can this
accelerometer distinguish between people
who have ms and people who do not
and what's really tricky about this
process is that we know that these
technologies are not static so there are
firmware updates there are hardware
updates there are new generations of
devices coming out constantly and so how
can we ensure that this
v3 framework is followed for all of
these different iterations of these
devices
and so really the take home message here
is that
these steps need to be redone when any
change is made to the biometric
monitoring technology and so if there
are changes to hardware if there are
changes to firmware which often occur
over air and many people don't know
about them that changes the data and so
it's very very important that that gets
taken into consideration when any of
these
biomets are being employed in practice
so that we know that they're fit for
purpose
so i want to switch gears for a moment
and talk about an implementation of a
digital biomarker so that we can see an
example of how this would actually be
useful um
so there was a lot of press around this
um sort of first
example of how we might be able to use
wearables to detect illness before a
person themselves might know that
they're sick
um so i always give the spoiler alert
before i actually explain the details of
what we did and so this was work that i
did alongside mike snyder
and his team at stanford
um where we um
collected a lot of data from a cohort of
people there were about a hundred
individuals wearing these smart watches
for
between
three and five years
um this was an interesting study because
a proportion of this population was
pre-diabetic um that's not what i'm
going to be talking about today today
i'm talking about infection detection
but that's initially why this study was
launched
um and so there were samples that were
given from these folks in the lab um
four times per year and then we also had
people come in to give samples when they
had anything kind of anomalous happen in
their lives so viral infections
stressful events dietary changes and so
what we can do with this is we can
capture what it looks like on a
biomolecular level to see health and to
see disease
so this was the setup for this study
where we developed these digital
biomarkers of infection
so
what i'm showing here is kind of just an
overview of what that data looks like we
have data from 43 people who are wearing
this intel basis watch it's measuring
three different things heart rate
temperature and accelerometry and you
can see here
that um showing in blue the skin
temperature data and then in these red
circles the heart rate data
these black squares here are showing the
number of days monitored
and so what you can immediately see is
that there's a big difference in the
variation between the skin temperature
measures and the heart rate measures
this makes a lot of sense because we
know that temperature cannot vary that
much and still sustain life
but heart rate we know can actually vary
dramatically between different people
and within a particular person
so this becomes challenging when we
think about how we can detect anomalies
within an individual
for example if we have two people let's
take this person here and this person
here that have very similar
resting heart rates
a measurement that's out here
for this person would be very anomalous
it does not fit within their
uh their sort of um confidence interval
of of their data whereas for this person
it would be very normal because it fits
within their typical
uh variation of their own body
so when we're developing these anomaly
detection algorithms it's very important
that we have a personalized look at this
and i can show this even a step further
if we look at some of the circadian
changes that occur
um with heart rate which i'm showing
here in pink or skin temperature which
i'm showing here in blue that at night
time we know when people are sleeping
their heart rate tends to dip down their
skin temperature tends to increase but
again this is true to different extents
for different people and even for some
people the opposite is true so it's
really important that when we're trying
to detect something that's abnormal in
an individual we know what normal looks
like for that particular person
so all of this is leading up to how do
we detect an infection in a person using
just smartwatch data
so the method that we developed here is
called the change of heart method and
the idea is that we have
um
outlier periods of time that we want to
try to detect from an individual
baseline so
the algorithm that we developed here
essentially goes based off of the
individual mean and standard deviation
and you can imagine that we could have
sliding windows where we take a a
sliding mean or a sliding standard
deviation and what we're looking for
are measurements that fall way outside
of what we deem to be normal
what's tricky about this is with a
smartwatch that measures one measurement
per second every day we get 86
400 measurements per sensor per day
the likelihood of any one of those
measurements falling well outside of the
norm is pretty high
um that's due to miss wear that's due to
all sorts of things that can happen and
so what we want to do instead is figure
out the proportion of measurements that
are falling outside of the norm so that
we're smoothing over these sort of
outlying events that are not due to a
change in physiology and so that's what
these the size of these circles is
showing essentially the size of this
circle is showing
how uh much the
the the proportion of measurements is
deviating from the baseline for that
particular person on that day
so employing this change of heart method
we were able to show in this this data
set from this ipod study at stanford
that um we were able to detect infection
in all of the participants where
infection occurred
what's tricky about this is that this
was a sort of real life study where
people were getting sick naturally and
so we didn't have an intervention where
people were actually infected
and and we kind of had to rely on what
naturally happens in the environment
um but we were able to develop these
these outlier detection algorithms that
did indicate when people had some sort
of influenza-like illness
and the way that this worked was that we
have these individual baselines based on
that we develop these digital biomarkers
and that leads us to this early
identification of acute and chronic
illness
so there's obviously a situation now
where there are a lot of people getting
sick
unfortunately we have a huge widespread
respiratory infection
um that is enabling us to to really see
in real time if we can detect these
signatures of infection from smart
watches alone and i'm showing here on
the screen just a sampling of studies
one of them is run by my group at duke
um cove identify but there are many
other studies that are going on in this
space trying to understand whether we
can actually detect infection before
people themselves know that they're sick
um and the idea there is that if you can
tell somebody they're sick even if
they're asymptomatic or pre-symptomatic
you can prevent them from going out and
spreading the virus so that is the goal
of this infection detection digital
biomarker work
all right so with the last couple
minutes here i want to talk about uh the
goal of digital biomarker discovery some
of the open source methods that we're
working on in this space and where all
of this can take us
um so so with digital biomarker
discovery really our goal is to extract
useful information and knowledge from
large volumes of biomet data in order to
improve
medical decision making or provide
insights for health decisions
and so some of the specific questions
that my group is looking at are can we
develop digital biomarkers to detect or
predict
improvements from sleep or exercise
pre-diabetes prior to symptoms
issues around infection how contagious a
person is or how susceptible they are to
contracting illness which becomes very
important if we think about sending out
warfighters into
a field setting so so knowing different
baseline physiology is really useful
we also are looking at circadian
destruction and shift workers and risk
of preterm birth and other women's
health issues
so what's interesting about this if we
think back to kind of the the beginning
uh slide that i mentioned where we are
developing these different digital
biomarkers a lot of these are using very
similar sensors and very similar machine
learning methods so there's a lot of
commonalities amongst these digital
biomarkers that we can capitalize on to
make the downstream process easier and
more standardized and for those of you
who have worked in the bioinformatics
room before potentially in genomics this
might feel reminiscent because a lot of
this is akin to the early days of
genomics where there there were a lot of
common methodologies being developed in
parallel and there was a realization
that
groups could actually come together and
do a better job and so we've been
working on this initiative called the
dbdp or the digital biomarker discovery
pipeline
which is an online environment that
aggregates open source software
algorithms data sets
and is working to develop
best practices for
developing new digital biomarkers and
evaluating them and so i'm just going to
throw this out there that you can go to
dbdp.org
and you can take a look this is actually
an older snapshot but
since since the snapshot we've added
more modules and actually our hope is
that the community continues to come
together and contributes more and more
modules to the dbdp
um right now this does require a level
of coding experience because all of the
digital biomarkers are written as code
modules on github but we do have some
really great documentation and resources
for people who are
new to
wearables or biomats or digital health
and wanna figure out where to get
started um i highly recommend checking
out the user guide and and some of our
sort of um getting started guides to
understand where to begin and and how to
become hopefully a long-term contributor
one of the other plugs that i'll push
here is as jen mentioned um there's a
huge need for work in privacy and
security
and that is an area where we really have
not done a lot and could really envision
some beautiful contributions
so
finally you know looking to the future
there's there are ever expanding options
of measurement for biomats um we have
smart textiles we have smart inc
um the ability to collect information
about our physiology and behaviors is
only going to increase as time goes on
so the hope is that this data gets used
for these four purposes um and and
nothing nefarious which is hopefully
what everyone here is working to ensure
but leading to patient empowerment
precision therapies just-in-time
interventions and improved access to
care
so i'll pop this last slide on here just
to say thanks to lots of folks who
contributed to some of the work that i
showed today
and i will turn it back over to jen
for a nice wrap-up
i had to do it at some point um i was
gonna say that was terrific and
um i'm lucky enough to sort of see your
work up close often but to see it sort
of presented and see the body of work
you've done is absolutely phenomenal let
me go ahead and
as you said round us out with a few
thoughts
um
what you showed us jessalyn was that you
know these technologies the measurement
capabilities using digital tools are
um not only extraordinary but they're
actually
largely ready for prime time the science
is there
but i
i think this place is at a place as us
at a crossroads right
how now we have these new tools in the
toolbox now we have these capabilities
how do we ensure that we actually use
them to improve individuals care right
do that um sort of precision medicine um
and not violate individuals privacy how
do we make sure that we're taking a
data-driven approach to public health
just like you showed us jessalyn and
make sure that we're not just
facilitating harmful surveillance how do
we make sure that we use these tools
that can extend by place that can extend
by capabilities that can connect
clinicians and experts with patients
wherever they are people during their
daily lives and make sure that we use
that to improve access and improve
health
equity and not just um sort of
exacerbate existing disparities
along the digital divide
and i think there are a few issues that
it's really important that we start to
address um some questions that we're
asking ourselves here at dime um that i
want to share with folks is you know
should we be thinking about broadband
access as a social determinant of health
we can see here a snapshot from the um
dartmouth atlas of help where you health
where you see red health is typically
better and where you see dark blue over
here we can see that broadband access is
much higher
these two areas correlate so if all we
do is roll these tools out to folks
where the technology infrastructure
already exists
we're simply helping the privileged few
rather than improving care for all
and this is something that extends well
beyond um sort of just broadband access
we have to think about other issues of
health equity digital literacy and other
social determinants of health and
there's a really terrific paper
that was published through a
collaboration between the ohio state
university and the national digital
inclusion alliance that i would highly
encourage everyone to take a look at the
link is here and it's absolutely
terrific
in our own work here at diamond
specifically through our collaborative
community data cc we're exploring the
idea of a new term we're thinking more
about less about sort of health
disparities it seems like a statement of
fact and trying to take us towards a
more action-oriented language let's
think more about digital inclusion let's
think you know and how that applies in
digital health measurement we would say
that it refers to activities and
practice throughout product development
deployment evaluation and
commercialization that are necessary not
optional but necessary to respond to the
needs of all populations
and to enable all individuals
communities to benefit from these
digital health measures again go back to
the power of these technologies that
jessalyn shared and think about how we
can um sort of deploy them
for everyone
data rights are increasingly important
we have to build trust in the health
data ecosystem and there's another
really terrific paper here around
privacy protections that encourage the
use of health relevant digital data
right and i think that's the other piece
too we are awash with data right now not
only are we making sure that these data
are being used to help not harm us how
are we making sure that we're not just
adding to the noise
there's a concept and actually we spoke
a little bit about this last year with
my colleague dina mendelson you know
said if this idea of redefining patient
safety in the digital era and this is
something we've considered we've
continued to explore here at dime the
definition of patient safety i actually
think is excellent
but there are new kinds of harm there
are new kinds of risks and the
definition of safety is contingent on
sort of current knowledge
of resources available and the context
in which care is delivered well current
knowledge is limited in this
interdisciplinary field clinicians
knowledge of cyber security for example
is limited we're not collaborating well
enough yet there isn't sufficient
knowledge sharing we need to do much
better at knowledge exchange in forums
like this um we need to think about all
of the different resources available
jessalyn mentioned some we produce a lot
of dime but there's a lot of silos of
resources unfortunately we also need to
think about new context of care care
isn't always delivered in the clinic
anymore and that means the data isn't
always confined to a locked filing
cabinet in the clinic either there are
all sorts of harms that now exist and we
want to make sure that we reap all of
the benefits of digital clinical
measures and limit those harms as far as
possible
there are other challenges around the
sort of um
lack of standardization the challenges
around integrating um these data into
the traditional clinical data sets
whether that's an edc an electronic data
capture system on the research side or
whether that's the electronic health
record on the care side
and perhaps one of the most important
things and something we focus on a lot
at dime is we have to build the digital
workforce necessary to actually support
the field we have to think about this
sort of big tent thinking to make sure
we have
all of the technical skills necessary
alongside the traditional clinical
competencies and importantly be making
sure that we're now attentive to the
patients we're all here to serve
this emerging um sort of group of
powerful citizen scientists and
ethicists are pretty critically
important in all of this work
we need to think about how we're
actually training the next generation
intentionally we can't hope is not a
strategy we really need to start
upstream
with the future workforce and we need to
do some re-scaling retooling of the
current workforce
and perhaps you know most importantly as
we think about making sure we're
ameliorating as far as possible health
disparities we have to have a commitment
to diversity in this workforce that goes
beyond just lip service on the technical
side of the shop on the clinical side of
the shop these workforces are not
terribly representative of the patients
we serve right now we have to make sure
as we combine them we improve that not
exacerbate um there's a sort of lack of
representation in the experts who are
working to develop digital health as a
field
i wanted to talk a little bit and the
last piece of data i'll present was from
an enormous collaboration that jessalyn
and the lab were part of and that we
were lucky enough to host the digital
clinical measures playbook um really
this is the essential guide for um sort
of the uh development and successful
development and deployment of digital
clinical measures across public health
across clinical care and across research
um and
what we sort of uh what we came up with
was a several hundred side playbook that
gets into every facet of thinking about
these measures from the sort of uh sort
of the v3 process that just uh jessalyn
mentioned um all the way through to the
ethical considerations and a variety of
different uh operational and technical
issues in between we really dive into
this
there are um also
110 i believe action-oriented tools from
educational curricula all the way
through to checklists and toolkits that
i encourage you to take a look at
and we also did some work really
thinking about the key archetypes the
key experts who need to be involved in
this process for it to be successful
we recognize that our academic
colleagues were a particularly important
cohort
in advancing the field of digital
clinical measures and making sure that
not only as we develop them but we
deploy them we can be confident that
they are trustworthy
um and so really there's some
fascinating data that i want to share in
closing this is data extracted from a
pre-print um currently um in a uh
gemma the journal of medical internet
research um and what you can see is the
kind of work that jessalyn discussed
around verification analytical
validation clinical validation that sort
of um that piece of the tech stack
they're doing great work there
importantly we're doing the work to make
sure that we're measuring things that
matter we're starting to get
sophisticated around how we
operationally
sort of deploy these
but there are some frankly alarming gaps
when we look at the last two years
of published literature
by academic researchers
as we can tie um security research
specifically to digital clinical
measures there was one study there was
only one published study really diving
into issues of data rights and
governance there was not a single paper
examining
their ethics of advancing these digital
clinical measures we also saw a death of
work trying to establish standards there
was limited work ensuring that these
things are economically feasible
and when we think about the operations
of deploying this in care vis-a-vis
research we saw that that was a lagging
sort of indicator too and if we think
back to how these measures are being
used in care right now without
reorganizing the workflow that shouldn't
be terribly surprising
and the last slide um before we close
out and i think that this gives us a
real call to action here is how are
these studies that are published how is
this research in the field of digital
clinical measures being funded
we can see that
there is robust government funding most
of it is going into operations of
research but we can see that government
funding is supporting these areas of v3
and identifying measures that matter
however
they are not supporting them in quite
the same way as we see other areas of
sort of medical research we're relying
heavily on independent foundations and
also on the hard work of academic
colleagues and labs like jessalyn to
find the resources internally from
investigative funds to do this work
we can see that 100 of the security
research is government funded but when
that's in one study that's not enough
it's critically important that we see
funding agencies start to identify
how important it is that we move to plug
this gap quickly we need to start having
requests for applications for grant
funding that address security that
address data rights and governance and
that address ethics and until we get to
that point the promise of this field to
deliver on everything it can bring to
improve health and health equity is
going to be limited by these risks
so with that that's why jail and i have
been so excited to come and join you
today this field is absolutely mission
critical to take all of this to scale so
with that i'll stop sharing i'm about to
get the sun right in my eyes i can see
it setting it's i'm about to go blind so
the timing is perfect thank you so much
for having us jessalyn would you like to
close out with any final remarks
oh this was great um thank you for this
community and hopefully this inspired
some um some interest in this work
because there's a lot that's needed
thanks
fantastic